Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,29 @@ These settings allow you to optimize performance for your specific infrastructur
- **Memory Constrained**: Reduce channel sizes (50-100 each)
- **High CPU**: Increase worker count up to 2x CPU cores

### Async Initial Loading Configuration (New!)
For faster connection setup and improved resilience with large datasets (>1000 companies/users):

- `USE_ASYNC_LOADING`: Enable asynchronous initial data loading (`true`/`false`, default: `false`)
- `ASYNC_LOADER_PAGE_SIZE`: Page size for initial data loading (default: `100`)
- `ASYNC_LOADER_CIRCUIT_BREAKER_THRESHOLD`: API call failures before circuit breaker opens (default: `5`)
- `ASYNC_LOADER_CIRCUIT_BREAKER_TIMEOUT`: Circuit breaker recovery timeout (default: `30s`)

**When to use Async Loading**:
- ✅ **Large datasets** (2000+ companies/users): Reduces connection time from ~60s to ~2s
- ✅ **Production environments**: WebSocket connects quickly, data loads in background
- ✅ **High availability**: Circuit breaker protects against API overload
- ❌ **Small datasets** (<500 companies/users): Synchronous loading is sufficient
- ❌ **Development**: Synchronous loading provides simpler debugging

Example for production with large datasets:
```bash
export USE_ASYNC_LOADING="true"
export ASYNC_LOADER_PAGE_SIZE="100"
export ASYNC_LOADER_CIRCUIT_BREAKER_THRESHOLD="5"
export ASYNC_LOADER_CIRCUIT_BREAKER_TIMEOUT="30s"
```

### Redis Configuration (Required)

**Note**: Redis is mandatory for the datastream replicator. The application will exit if it cannot connect to Redis.
Expand Down
Loading