Salesforce Streaming Application - Technical Design Document

1. System Overview

1.1 Purpose

The application provides real-time streaming of Salesforce Change Data Capture (CDC) and Platform Events using Node.js and WebSocket technology. It allows users to:

• Authenticate with Salesforce using OAuth2
• Subscribe to CDC or Platform Events
• View real-time event updates with historical data
• Maintain persistent connection for live updates

1.2 Architecture Diagram

graph TD
    subgraph Client
        UI[Web Browser]
        WS_Client[WebSocket Client]
    end
    
    subgraph Server
        Express[Express Server]
        WS_Server[WebSocket Server]
        SF_Service[Salesforce Service]
        Auth[Auth Controller]
        Stream[Stream Controller]
    end
    
    subgraph Salesforce
        SF_Auth[OAuth2 Endpoint]
        SF_Stream[Streaming API]
        SF_CDC[CDC Events]
        SF_Platform[Platform Events]
    end
    
    UI --> Express
    WS_Client <--> WS_Server
    Express --> Auth
    Express --> Stream
    Auth <--> SF_Auth
    Stream --> SF_Service
    SF_Service <--> SF_Stream
    SF_Stream --> SF_CDC
    SF_Stream --> SF_Platform
    WS_Server --> UI

Loading

2. Technical Stack

2.1 Core Technologies

• Backend: Node.js v16+
• Framework: Express.js v4.18
• Template Engine: EJS v3.1
• WebSocket: ws v8.13
• Salesforce Connection: JSForce v1.11
• Session Management: express-session v1.17
• Environment Variables: dotenv v16.0

2.2 Development Dependencies

{
  "dependencies": {
    "dotenv": "^16.0.3",
    "ejs": "^3.1.9",
    "express": "^4.18.2",
    "express-session": "^1.17.3",
    "jsforce": "^1.11.1",
    "ws": "^8.13.0"
  }
}

3. Component Details

3.1 Directory Structure

├── .env
├── app.js
├── package.json
├── public/
│   └── css/
│       └── style.css
├── routes/
│   ├── auth.js
│   └── stream.js
├── controllers/
│   ├── authController.js
│   └── streamController.js
├── views/
│   ├── login.ejs
│   └── stream.ejs
└── services/
    └── salesforceService.js

3.2 Component Responsibilities

3.2.1 Server Components

app.js

• Entry point for the application
• Express and WebSocket server configuration
• Middleware setup
• Route registration
• WebSocket connection management

routes/auth.js

• OAuth2 authentication routes
• Login endpoint
• Callback handling
• Session management

routes/stream.js

• Streaming subscription routes
• Event handling endpoints
• WebSocket event broadcasting

controllers/authController.js

• Salesforce OAuth2 flow implementation
• Token management
• User session handling

controllers/streamController.js

• CDC and Platform Event subscription logic
• Event data processing
• WebSocket broadcast management

services/salesforceService.js

• Salesforce API integration
• Event subscription management
• Data transformation
• Error handling

3.2.2 Client Components

views/login.ejs

• Login page UI
• Salesforce authentication button
• Initial user interface

views/stream.ejs

• Main streaming interface
• Event type selection
• Event display container
• WebSocket client implementation

public/css/style.css

• Application styling
• Event container design
• Animation definitions
• Responsive layout

4. Data Flow

4.1 Authentication Flow

1. User clicks "Login with Salesforce"
2. System redirects to Salesforce OAuth2 endpoint
3. User authenticates with Salesforce
4. Salesforce redirects to callback URL
5. System stores access token in session
6. User is redirected to streaming page

4.2 Event Streaming Flow

1. User selects event type and name
2. Client establishes WebSocket connection
3. Server subscribes to Salesforce topic
4. Salesforce sends events (historical + real-time)
5. Server broadcasts events to all connected clients
6. Clients display events with appropriate styling

sequenceDiagram
    participant Client
    participant Server
    participant WebSocket
    participant Salesforce
    
    Client->>Server: Subscribe to events
    Server->>Salesforce: Create streaming topic subscription
    Client->>WebSocket: Establish connection
    Salesforce->>Server: Send historical events
    Server->>WebSocket: Broadcast events
    WebSocket->>Client: Display events
    loop Real-time Updates
        Salesforce->>Server: New event occurs
        Server->>WebSocket: Broadcast new event
        WebSocket->>Client: Update UI
    end

Loading

5. Configuration

5.1 Environment Variables

SF_CLIENT_ID=your_client_id
SF_CLIENT_SECRET=your_client_secret
SF_REDIRECT_URI=http://localhost:3000/auth/callback
SF_LOGIN_URL=https://login.salesforce.com

5.2 Salesforce Configuration

1. Create Connected App in Salesforce
2. Configure OAuth settings
3. Enable relevant CDC or Platform Events
4. Set appropriate permissions

6. Error Handling

6.1 WebSocket Reconnection

• Maximum 5 reconnection attempts
• 3-second delay between attempts
• Automatic cleanup of dead connections

6.2 Error Types

1. Authentication Errors

   • Invalid credentials
   • Token expiration
   • Permission issues

2. Subscription Errors

   • Invalid event type
   • Topic not found
   • Permission denied

3. Connection Errors

   • WebSocket disconnection
   • Network issues
   • Server timeout

7. Security Considerations

7.1 Authentication

• OAuth2 secure flow
• Session-based authentication
• HTTPS recommended for production

7.2 Data Protection

• No storage of Salesforce credentials
• Session-only token storage
• Secure WebSocket connection

8. Performance Considerations

8.1 WebSocket

• Connection pooling
• Event buffering
• Client-side throttling

8.2 Event Processing

• Asynchronous event handling
• Batch processing for historical events
• Memory management for large event volumes

9. Deployment Requirements

9.1 Server Requirements

• Node.js v16+
• 512MB RAM minimum
• Network access to Salesforce APIs

9.2 Client Requirements

• Modern web browser with WebSocket support
• JavaScript enabled
• Stable internet connection

10. Testing Strategy

10.1 Test Areas

1. Authentication Flow
2. WebSocket Connection
3. Event Subscription
4. Data Display
5. Error Handling
6. Reconnection Logic

10.2 Test Types

• Unit Tests
• Integration Tests
• End-to-End Tests
• Performance Tests

11. Maintenance and Monitoring

11.1 Monitoring Points

• WebSocket connection status
• Event processing time
• Error rates
• Memory usage
• Connection count

11.2 Logging

• Authentication attempts
• Connection events
• Subscription status
• Error details
• Performance metrics