October Backend

Try it on the Octob3r platform: https://october.circuit-x.com

🎥 Watch the demo video

📚 Documentation

Quick Start Guide - Development workflow and commit conventions
Contributing Guide - Conventional commits specification and examples
AWS Deployment Guide - Complete AWS setup and deployment
Release Management Guide - Automated releases and versioning

A robust Go server built following NASA's "Power of 10" rules for clean and safe code, featuring MongoDB integration, company data management, and rate-limited APIs.

NASA Clean Code Compliance

This application follows NASA's coding standards for critical systems:

Avoid complex flow constructs - No goto statements, setjmp, or recursion
No dynamic memory allocation - All memory allocations are at startup
No functions larger than 60 lines - All functions are kept simple and focused
Return value checking - All function return values are checked
Limited scope - Variables have minimal scope
Runtime assertions - Critical assumptions are verified
Restricted preprocessor use - Minimal macro usage
Limited pointer use - Careful pointer management
Compile with warnings - All warnings treated as errors
Static analysis - Code is regularly analyzed for issues

Features

Core Infrastructure

Graceful Shutdown: Proper signal handling and resource cleanup
Structured Logging: JSON-formatted logs with context
Configuration Management: Environment-based configuration with validation
Error Handling: Comprehensive error handling and recovery
Health Checks: Multiple monitoring endpoints (/health, /liveness, /readiness)
Metrics Collection: Prometheus-style metrics with JSON export
Middleware: Request logging, recovery, metrics tracking, and security middleware
Timeouts: Proper timeout handling for all operations
AWS Deployment: Production deployment on ECS Fargate with ALB and custom domain

Database Integration

MongoDB Support: Full MongoDB integration with connection pooling
Company Management: CRUD operations for defense/aerospace companies
News Processing: Automated RSS feed processing and article storage
Data Validation: Comprehensive input validation and sanitization
Indexing: Optimized database indexes for performance

News & RSS Features

RSS Feed Processing: Automated collection from company feeds
Article Management: Storage with deduplication and validation
Sentiment Analysis: Basic sentiment scoring for articles
Relevance Scoring: Company relevance calculation
Filtering: Advanced filtering by company, date, sentiment, and relevance
Pagination: Efficient pagination for large datasets

API Features

Rate Limiting: Token bucket algorithm with per-IP tracking (10 req/s, burst 20)
RESTful Endpoints: Clean REST API design
Error Responses: Consistent error response format
Request Logging: Detailed request/response logging
Market Data Integration: Real-time stock quotes via Finnhub API
CORS Support: Cross-origin resource sharing enabled
Metrics Export: Prometheus-compatible metrics in text and JSON formats

AI/RAG Features

Natural Language Queries: Ask questions in plain English about companies
OpenAI Integration: Powered by GPT-4o-mini for cost-effective AI responses
Retrieval-Augmented Generation: Responses backed by real news articles
Web Search Integration: Google Custom Search for company-related topics when database context is insufficient
Article Summarization: AI-powered summarization of individual news articles
Memory Caching: In-memory cache for AI summaries to reduce API costs and improve performance
Cache Statistics: Monitor cache hit/miss rates and performance
Company-Based Validation: Web search allowed for ANY question about companies in our database
Query Analysis: Intelligent parsing of user intent and entities
Source Attribution: See which articles and web sources were used for each response
Confidence Scoring: Reliability assessment for AI-generated answers
Company Context: Focus queries on specific companies in the database
AI-Powered Filtering: Let OpenAI's intelligence handle response appropriateness

Quick Start

Prerequisites

Go 1.21 or later
MongoDB 4.4 or later
OpenAI API key (for AI/RAG features)
Google Custom Search API key and Engine ID (for web search)
Finnhub API key (for market data features)

Installation

Clone the repository
Install dependencies:
```
go mod download
```

Set up environment:

cp .env.example .env
# Edit .env with your configuration, including:
# OPENAI_API_KEY=your_openai_api_key_here
# CUSTOM_SEARCH_API_KEY=your_google_api_key_here
# CUSTOM_SEARCH_ENGINE_ID=your_search_engine_id_here
# FINNHUB_API_KEY=your_finnhub_api_key_here

Set up conventional commit rules (recommended):
```
./scripts/setup-commit-rules.sh
```
This configures your local git to use commit message templates and validation.

Start MongoDB (using Docker):

docker run -d --name mongodb -p 27017:27017 mongo:latest

Build and run:

make build
make seed-data  # Seed initial company data
make run

Build and run:

make build
make seed-data  # Seed initial company data
make run

API Endpoints

Health & Monitoring

Health Check

GET /health

Basic health check endpoint.

Example:

curl https://oct.circuit-x.com/health

Liveness Probe

GET /liveness

Kubernetes-style liveness probe (minimal check).

Readiness Probe

GET /readiness

Kubernetes-style readiness probe (checks dependencies).

Metrics (Prometheus format)

GET /metrics

Prometheus-style metrics in text format.

Metrics (JSON format)

GET /metrics.json

Detailed metrics in JSON format including request counts, latencies, and cache statistics.

Example:

curl https://oct.circuit-x.com/metrics.json

Company API

Get All Companies

GET /companies

Rate Limited: 10 requests/second, burst of 20

Example:

curl https://oct.circuit-x.com/companies

Get Company by Name

GET /company/{company-name}

Rate Limited: 10 requests/second, burst of 20

Examples:

# Get Lockheed Martin
curl https://oct.circuit-x.com/company/Lockheed%20Martin

# Get Raytheon Technologies  
curl https://oct.circuit-x.com/company/Raytheon%20Technologies

News API

Get News Articles

GET /news

Rate Limited: 10 requests/second, burst of 20

Query Parameters:

company: Filter by company name
start_date: Filter from date (YYYY-MM-DD)
end_date: Filter until date (YYYY-MM-DD)
sentiment: Filter by sentiment (-2 to 2)
min_relevance: Minimum relevance score (0.0 to 1.0)
limit: Number of results (default: 50, max: 1000)
offset: Pagination offset

Examples:

# Get recent news for Lockheed Martin
curl "https://oct.circuit-x.com/news?company=Lockheed%20Martin&limit=10"

# Get positive news from last month
curl "https://oct.circuit-x.com/news?sentiment=1&start_date=2024-09-23&end_date=2024-10-23"

# Get high relevance news with pagination
curl "https://oct.circuit-x.com/news?min_relevance=0.8&limit=20&offset=40"

Get Specific Article

GET /news/{id}

Example:

curl https://oct.circuit-x.com/news/507f1f77bcf86cd799439011

Get News by Company

GET /news/company/{name}

Rate Limited: 10 requests/second, burst of 20

Example:

curl https://oct.circuit-x.com/news/company/Lockheed%20Martin

AI/RAG API

Ask AI Questions

POST /ai/query

Rate Limited: 10 requests/second, burst of 20

Request Body:

{
  "question": "How did RTX perform this quarter?",
  "company_context": ["Raytheon Technologies"]
}

Examples:

# Financial performance query
curl -X POST https://oct.circuit-x.com/ai/query \
  -H "Content-Type: application/json" \
  -d '{"question": "How did RTX perform this quarter?"}'

# Defense contracts query
curl -X POST https://oct.circuit-x.com/ai/query \
  -H "Content-Type: application/json" \
  -d '{"question": "What defense contracts did RTX recently win?"}'

# Military developments query
curl -X POST https://oct.circuit-x.com/ai/query \
  -H "Content-Type: application/json" \
  -d '{"question": "What are the latest military training developments?", "company_context": ["US War Department"]}'

Analyze Query Intent

POST /ai/analyze

Request Body:

{
  "question": "What were RTX earnings this quarter?"
}

Example:

curl -X POST https://oct.circuit-x.com/ai/analyze \
  -H "Content-Type: application/json" \
  -d '{"question": "What were RTX earnings this quarter?"}'

Web Search for Defense Topics

POST /ai/web-search

Request Body:

{
  "question": "Latest defense contracts for Raytheon",
  "companies": ["Raytheon Technologies"]
}

Example:

curl -X POST https://oct.circuit-x.com/ai/web-search \
  -H "Content-Type: application/json" \
  -d '{"question": "Latest defense contracts for Raytheon", "companies": ["Raytheon Technologies"]}'

Note: Web search is allowed for ANY question about companies that exist in our database. OpenAI's intelligence handles response appropriateness.

Summarize Article

GET /ai/summarise/{articleId}

Rate Limited: 10 requests/second, burst of 20

Generate an AI-powered summary of a specific news article. Results are cached in memory to reduce API costs.

Example:

curl https://oct.circuit-x.com/ai/summarise/507f1f77bcf86cd799439011

Response:

{
  "article_id": "507f1f77bcf86cd799439011",
  "summary": "Lockheed Martin announced Q3 earnings showing...",
  "key_points": [
    "Revenue increased by 12%",
    "New defense contract worth $2.5B",
    "F-35 production ramping up"
  ],
  "sentiment": "positive",
  "cached": false,
  "generated_at": "2024-10-23T10:30:00Z"
}

Cache Statistics

GET /ai/cache/stats

Rate Limited: 10 requests/second, burst of 20

Get statistics about the AI summary cache performance.

Example:

curl https://oct.circuit-x.com/ai/cache/stats

Response:

{
  "total_items": 145,
  "hits": 892,
  "misses": 145,
  "hit_rate": 0.86,
  "evictions": 12
}

Market Data API

Real-time stock market data powered by Finnhub.

Get Stock Quote

GET /market/quote/{ticker}

Rate Limited: 10 requests/second, burst of 20

Example:

# Get Raytheon Technologies quote
curl https://oct.circuit-x.com/market/quote/RTX

Response:

{
  "ticker": "RTX",
  "current_price": 92.45,
  "change": 1.23,
  "percent_change": 1.35,
  "high": 93.12,
  "low": 91.87,
  "open": 92.00,
  "previous_close": 91.22,
  "timestamp": "2024-10-23T16:00:00Z"
}

Get Multiple Quotes

GET /market/quotes?tickers=RTX,LMT,NOC

Rate Limited: 10 requests/second, burst of 20

Query Parameters:

tickers: Comma-separated list of ticker symbols

Example:

curl "https://oct.circuit-x.com/market/quotes?tickers=RTX,LMT,NOC"

Get Available Tickers

GET /market/tickers

Rate Limited: 10 requests/second, burst of 20

Returns all available tickers from companies in the database.

Example:

curl https://oct.circuit-x.com/market/tickers

Response:

{
  "tickers": [
    {
      "ticker": "RTX",
      "company": "Raytheon Technologies",
      "industry": "Aerospace"
    },
    {
      "ticker": "LMT",
      "company": "Lockheed Martin",
      "industry": "Defense"
    }
  ],
  "count": 2
}

Get Market Status

GET /market/status/{exchange}

Rate Limited: 10 requests/second, burst of 20

Example:

curl https://oct.circuit-x.com/market/status/US

Response:

{
  "exchange": "US",
  "is_open": true,
  "status": "open",
  "timezone": "America/New_York",
  "current_time": "2024-10-23T14:30:00Z"
}

Pre-loaded Companies

The system includes two defense/aerospace companies:

Lockheed Martin (LMT)
- Industry: Defense
- Feed: https://news.lockheedmartin.com/rss
- Employees: 116,000
Raytheon Technologies (RTX)
- Industry: Aerospace
- Feed: https://www.rtx.com/rss-feeds/news
- Employees: 185,000

RSS Feed Processing

The application includes automated RSS feed processing to collect and store news articles:

Automatic Background Refresh

The server automatically processes RSS feeds every 2 hours in the background. This happens:

Immediately on application startup
Every 2 hours thereafter
For all companies in the database

No manual intervention needed for production deployments!

Manual Processing Commands

# Seed company data first
make seed-data

# Process all company RSS feeds
make process-feeds

# Process specific company feed
make process-feed COMPANY="Lockheed Martin"

# Direct command usage
./bin/feed-processor
./bin/feed-processor -company="Raytheon Technologies"

Article Processing Features

Deduplication: Articles are deduplicated using GUID or URL
Sentiment Analysis: Basic sentiment scoring (-2 to +2)
Relevance Scoring: Company relevance calculation (0.0 to 1.0)
Automatic Indexing: Database indexes for optimal query performance

Architecture

Project Structure

cmd/
├── api/main.go              # Application entry point
├── seed/main.go             # Database seeding utility
└── feed-processor/main.go   # RSS feed processor
config/                      # Configuration management
pkg/
├── logger/                  # Structured logging
└── metrics/                 # Metrics collection
internal/
├── domain/
│   ├── ai/                  # AI/RAG business logic
│   ├── company/             # Company business logic
│   ├── market/              # Market data business logic
│   ├── news/                # News business logic
│   └── shared/              # Shared domain models
├── infra/
│   ├── ai/                  # OpenAI integration
│   ├── cache/               # In-memory caching
│   ├── database/mongodb/    # MongoDB implementations
│   ├── feed/                # RSS feed processing
│   ├── market/              # Finnhub integration
│   └── search/              # Google Custom Search
└── interfaces/http/
    ├── handlers/            # HTTP request handlers
    ├── middleware/          # HTTP middleware (logging, metrics, rate limiting)
    └── router.go            # Route definitions

System Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Internet / Users                        │
└───────────────────────────┬─────────────────────────────────┘
                            │
                            │ HTTPS (443) / HTTP (80)
                            ▼
              ┌─────────────────────────────┐
              │   Route53 DNS               │
              │   october.circuit-x.com     │
              └──────────────┬──────────────┘
                             │
                             ▼
              ┌─────────────────────────────┐
              │  Application Load Balancer  │
              │  (october-alb)              │
              │  - Health checks            │
              │  - SSL termination          │
              └──────────────┬──────────────┘
                             │
                             ▼
       ┌─────────────────────────────────────────┐
       │     ECS Fargate (october-cluster)       │
       │  ┌───────────────────────────────────┐  │
       │  │  October Backend Container        │  │
       │  │  - Go 1.21                        │  │
       │  │  - Port 8080                      │  │
       │  │  - Auto-scaling enabled           │  │
       │  └───────────┬───────────────────────┘  │
       └──────────────┼──────────────────────────┘
                      │
          ┌───────────┼───────────┐
          │           │           │
          ▼           ▼           ▼
    ┌─────────┐ ┌─────────┐ ┌──────────┐
    │ MongoDB │ │ OpenAI  │ │ Finnhub  │
    │ Atlas   │ │   API   │ │   API    │
    └─────────┘ └─────────┘ └──────────┘
          │           │           │
          │           │           └── Market Data
          │           └────────────── AI/RAG
          └────────────────────────── Data Storage

    External Integrations:
    - Google Custom Search API (Web search)
    - CloudWatch Logs (Monitoring)
    - SSM Parameter Store (Secrets)

Running the Application

Prerequisites

Go 1.21 or later
MongoDB (if using database features)

Environment Setup

Copy environment template:
```
cp .env.example .env
```
Adjust configuration in .env as needed

Build and Run

# Build the application
go build -o october-server ./cmd/api

# Run the application
./october-server

Development

# Run directly
go run ./cmd/api

# Run with custom log level
LOG_LEVEL=debug go run ./cmd/api

# Run with custom port
SERVER_PORT=9090 go run ./cmd/api

# Run with live reload (requires air)
make dev

# Test all API endpoints (server must be running)
make test-api

# Test AI/RAG endpoints (server must be running)
make test-ai

Health Check

The application provides a health check endpoint:

curl http://localhost:8080/health

Response:

{
  "status": "healthy",
  "timestamp": "2024-10-23T10:30:00Z"
}

Configuration

All configuration is handled through environment variables:

Variable	Default	Description
`SERVER_HOST`	`0.0.0.0`	Server bind address
`SERVER_PORT`	`8080`	Server port
`SERVER_READ_TIMEOUT`	`15s`	HTTP read timeout
`SERVER_WRITE_TIMEOUT`	`15s`	HTTP write timeout
`SERVER_IDLE_TIMEOUT`	`60s`	HTTP idle timeout
`DATABASE_URI`	`mongodb://localhost:27017/october`	MongoDB connection string
`LOG_LEVEL`	`info`	Log level (debug, info, warn, error)
`OPENAI_API_KEY`	-	OpenAI API key for AI/RAG features (required)
`CUSTOM_SEARCH_API_KEY`	-	Google Custom Search API key (required)
`CUSTOM_SEARCH_ENGINE_ID`	-	Google Custom Search Engine ID (required)
`FINNHUB_API_KEY`	-	Finnhub API key for market data (required)
`AWS_ACCESS_KEY_ID`	-	AWS access key (for deployment only)
`AWS_SECRET_ACCESS_KEY`	-	AWS secret key (for deployment only)

Safety Features

Error Handling

All functions check return values
Comprehensive error logging
Graceful degradation on failures

Resource Management

Proper cleanup on shutdown
Connection pooling and limits
Memory usage monitoring

Signal Handling

SIGINT and SIGTERM handling
Graceful shutdown with timeout
Active connection draining

Logging

Structured JSON logging
Request/response logging
Error tracking and alerting

Development Guidelines

Function Size: Keep functions under 60 lines
Error Handling: Always check and handle errors
Resource Cleanup: Use defer for cleanup operations
Testing: Write tests for all public functions
Documentation: Document all exported functions and types
Validation: Validate all inputs and configurations

Production Deployment

AWS ECS Fargate (Recommended)

The application is production-ready and deployed on AWS using:

ECS Fargate: Serverless container orchestration
Application Load Balancer: Traffic distribution and health checks
Route53: DNS management with custom domain
ACM: Free SSL/TLS certificates
CloudWatch: Centralized logging and monitoring
SSM Parameter Store: Secure secrets management

Production URL: https://oct.circuit-x.com

For detailed deployment instructions, see DEPLOYMENT.md

CI/CD & Release Management

The repository includes automated GitHub Actions workflows:

Commit Validation Workflow

Trigger: Push to any branch or Pull Request
Actions:
- Validates all commit messages follow Conventional Commits specification
- Provides detailed feedback on invalid messages
- Posts PR comments with examples and fix instructions
File: .github/workflows/commit-lint.yml

Deployment Workflow

Trigger: Push to main branch
Actions:
- Build and push Docker image to ECR
- Deploy to ECS Fargate
- Configure custom domain with HTTPS
File: .github/workflows/deploy.yml

Release Workflow

Trigger: Successful deployment completion
Actions:
- Automatically determines version bump (major/minor/patch) based on commit messages
- Creates Git tag
- Generates release notes with categorized changes
- Creates GitHub release with production links
File: .github/workflows/release.yml
File: .github/workflows/release.yml

Commit Message Conventions for version bumping:

feat: or feature: → Minor version bump (e.g., v1.0.0 → v1.1.0)
fix: or bugfix: → Patch version bump (e.g., v1.0.0 → v1.0.1)
feat!: or BREAKING CHANGE: → Major version bump (e.g., v1.0.0 → v2.0.0)
Other commits → Patch version bump

Docker (Alternative)

FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY . .
RUN go mod tidy && go build -o october-server ./cmd/api

FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /app/october-server .
CMD ["./october-server"]

Build and run:

make docker-build
make docker-run

System Service

Create a systemd service file for production deployment:

[Unit]
Description=October Backend Service
After=network.target

[Service]
Type=simple
User=october
Group=october
WorkingDirectory=/opt/october
ExecStart=/opt/october/october-server
Restart=always
RestartSec=5
EnvironmentFile=/opt/october/.env

[Install]
WantedBy=multi-user.target

Monitoring

The application provides comprehensive monitoring capabilities:

Health Endpoints

Health: GET /health - Basic health check
Liveness: GET /liveness - Kubernetes-style liveness probe
Readiness: GET /readiness - Kubernetes-style readiness probe

Metrics Endpoints

Prometheus: GET /metrics - Prometheus-compatible text format
JSON Metrics: GET /metrics.json - Detailed JSON metrics including:
- Request counts by endpoint and status code
- Request latencies (p50, p90, p95, p99)
- Active connections
- Uptime
- Memory usage
- Cache statistics (hits, misses, hit rate)
- API response times

Structured Logging

JSON-formatted logs with contextual information
Request/response logging with duration
Error tracking with stack traces
Configurable log levels (debug, info, warn, error)

Production Monitoring

CloudWatch Logs: Centralized log aggregation
CloudWatch Metrics: Custom metrics and alarms
ALB Health Checks: Continuous availability monitoring
ECS Service Monitoring: Container health and auto-recovery

Security

Input validation on all endpoints
Proper error message sanitization
Rate limiting (when configured)
CORS protection
Security headers middleware

Name		Name	Last commit message	Last commit date
Latest commit History 64 Commits
.github/workflows		.github/workflows
aws		aws
cmd		cmd
config		config
docker		docker
docs		docs
internal		internal
pkg		pkg
scripts		scripts
.dockerignore		.dockerignore
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
.gitignore.env		.gitignore.env
.gitmessage		.gitmessage
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile		Dockerfile
Makefile		Makefile
README.md		README.md
docker-compose.yml		docker-compose.yml
go.mod		go.mod
go.sum		go.sum

Neph-dev/october_backend

Folders and files

Latest commit

History

Repository files navigation

October Backend

Try it on the Octob3r platform: https://october.circuit-x.com

🎥 Watch the demo video

📚 Documentation

NASA Clean Code Compliance

Features

Core Infrastructure

Database Integration

News & RSS Features

API Features

AI/RAG Features

Quick Start

Prerequisites

Installation

API Endpoints

Health & Monitoring

Health Check

Liveness Probe

Readiness Probe

Metrics (Prometheus format)

Metrics (JSON format)

Company API

Get All Companies

Get Company by Name

News API

Get News Articles

Get Specific Article

Get News by Company

AI/RAG API

Ask AI Questions

Analyze Query Intent

Web Search for Defense Topics

Summarize Article

Cache Statistics

Market Data API

Get Stock Quote

Get Multiple Quotes

Get Available Tickers

Get Market Status

Pre-loaded Companies

RSS Feed Processing

Automatic Background Refresh

Manual Processing Commands

Article Processing Features

Architecture

Project Structure

System Architecture

Running the Application

Prerequisites

Environment Setup

Build and Run

Development

Health Check

Configuration

Safety Features

Error Handling

Resource Management

Signal Handling

Logging

Development Guidelines

Production Deployment

AWS ECS Fargate (Recommended)

CI/CD & Release Management

Commit Validation Workflow

Deployment Workflow

Release Workflow

Docker (Alternative)

System Service

Monitoring

Health Endpoints

Metrics Endpoints

Structured Logging

Production Monitoring

Security

About

Packages