Skip to content

Streaming Responses

Jump to bottom
Rumen Damyanov edited this page Oct 3, 2025 · 1 revision

Streaming Responses Guide

This guide covers everything you need to know about using streaming responses with php-chatbot.

Table of Contents

Overview

Streaming responses allow your chatbot to send replies as they are generated, rather than waiting for the complete response. This creates a more interactive and responsive user experience, similar to how ChatGPT displays responses.

Benefits:

  • Better UX: Users see responses appearing in real-time
  • Perceived Performance: Feels faster even though total time may be similar
  • Progressive Disclosure: Long responses are more digestible
  • Modern Experience: Matches expectations from popular AI interfaces

Supported Models

Provider Streaming Support Notes
OpenAI ✅ Yes All models (GPT-4, GPT-3.5, etc.)
Anthropic ✅ Yes All Claude models
Google Gemini ✅ Yes All Gemini models
xAI ✅ Yes All Grok models
Meta ✅ Yes All LLaMA models
DeepSeek ❌ No Coming soon
Ollama ❌ No Coming soon
DefaultAiModel ❌ No Fallback model doesn't support streaming

Basic Usage

Simple Streaming Example

<?php
use Rumenx\PhpChatbot\PhpChatbot;
use Rumenx\PhpChatbot\Models\OpenAiModel;

// Create model and chatbot
$model = new OpenAiModel($_ENV['OPENAI_API_KEY']);
$chatbot = new PhpChatbot($model);

// Stream response
foreach ($chatbot->askStream('Explain quantum computing in simple terms') as $chunk) {
    echo $chunk;
    flush(); // Important: Send to output immediately
}

Checking Streaming Support

Always check if a model supports streaming before using askStream():

<?php
use Rumenx\PhpChatbot\Contracts\StreamableModelInterface;

if ($model instanceof StreamableModelInterface && $model->supportsStreaming()) {
    // Use streaming
    foreach ($chatbot->askStream($message) as $chunk) {
        echo $chunk;
    }
} else {
    // Fallback to regular response
    $reply = $chatbot->ask($message);
    echo $reply;
}

With Context Parameters

Streaming respects all context parameters:

<?php
$context = [
    'prompt' => 'You are a helpful coding assistant.',
    'temperature' => 0.7,
    'max_tokens' => 500
];

foreach ($chatbot->askStream('How do I write a PHP function?', $context) as $chunk) {
    echo $chunk;
}

Backend Implementation

Laravel Controller with Streaming

<?php
namespace App\Http\Controllers;

use Illuminate\Http\Request;
use Rumenx\PhpChatbot\PhpChatbot;
use Rumenx\PhpChatbot\Models\ModelFactory;
use Symfony\Component\HttpFoundation\StreamedResponse;

class ChatbotController extends Controller
{
    public function stream(Request $request): StreamedResponse
    {
        $message = $request->input('message');
        
        $config = config('phpchatbot');
        $model = ModelFactory::make($config);
        $chatbot = new PhpChatbot($model, $config);
        
        return new StreamedResponse(function () use ($chatbot, $message) {
            // Set SSE headers
            header('Content-Type: text/event-stream');
            header('Cache-Control: no-cache');
            header('X-Accel-Buffering: no'); // Disable nginx buffering
            
            try {
                foreach ($chatbot->askStream($message) as $chunk) {
                    echo "data: " . json_encode(['chunk' => $chunk]) . "\n\n";
                    flush();
                }
                
                echo "data: [DONE]\n\n";
                flush();
            } catch (\Exception $e) {
                echo "data: " . json_encode(['error' => $e->getMessage()]) . "\n\n";
                flush();
            }
        });
    }
}

Symfony Controller with Streaming

<?php
namespace App\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\StreamedResponse;
use Rumenx\PhpChatbot\PhpChatbot;
use Rumenx\PhpChatbot\Models\ModelFactory;

class ChatbotController extends AbstractController
{
    public function stream(Request $request): StreamedResponse
    {
        $message = $request->request->get('message');
        
        $config = require $this->getParameter('kernel.project_dir').'/src/Config/phpchatbot.php';
        $model = ModelFactory::make($config);
        $chatbot = new PhpChatbot($model, $config);
        
        return new StreamedResponse(function () use ($chatbot, $message) {
            header('Content-Type: text/event-stream');
            header('Cache-Control: no-cache');
            header('X-Accel-Buffering: no');
            
            foreach ($chatbot->askStream($message) as $chunk) {
                echo "data: " . json_encode(['chunk' => $chunk]) . "\n\n";
                flush();
            }
            
            echo "data: [DONE]\n\n";
            flush();
        });
    }
}

Plain PHP Endpoint

<?php
// api/stream.php
require_once __DIR__ . '/../vendor/autoload.php';

use Rumenx\PhpChatbot\PhpChatbot;
use Rumenx\PhpChatbot\Models\OpenAiModel;

// Set SSE headers
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
header('Connection: keep-alive');
header('X-Accel-Buffering: no');

// Disable output buffering
@ini_set('output_buffering', 'off');
@ini_set('zlib.output_compression', false);

$message = $_POST['message'] ?? 'Hello';

$model = new OpenAiModel($_ENV['OPENAI_API_KEY']);
$chatbot = new PhpChatbot($model);

try {
    foreach ($chatbot->askStream($message) as $chunk) {
        echo "data: " . json_encode(['chunk' => $chunk]) . "\n\n";
        
        if (ob_get_level() > 0) {
            ob_flush();
        }
        flush();
    }
    
    echo "data: [DONE]\n\n";
    flush();
} catch (Exception $e) {
    echo "data: " . json_encode(['error' => $e->getMessage()]) . "\n\n";
    flush();
}

Frontend Integration

Vanilla JavaScript (EventSource)

function streamChatResponse(message) {
    const chatDisplay = document.getElementById('chat-response');
    chatDisplay.textContent = '';
    
    const eventSource = new EventSource(`/api/chatbot/stream?message=${encodeURIComponent(message)}`);
    
    eventSource.onmessage = function(event) {
        if (event.data === '[DONE]') {
            eventSource.close();
            console.log('Stream completed');
            return;
        }
        
        try {
            const data = JSON.parse(event.data);
            
            if (data.error) {
                console.error('Stream error:', data.error);
                chatDisplay.textContent += '\n[Error: ' + data.error + ']';
                eventSource.close();
                return;
            }
            
            if (data.chunk) {
                chatDisplay.textContent += data.chunk;
            }
        } catch (e) {
            console.error('Parse error:', e);
        }
    };
    
    eventSource.onerror = function(error) {
        console.error('EventSource error:', error);
        eventSource.close();
    };
}

// Usage
streamChatResponse('What is machine learning?');

React Component

import React, { useState, useEffect, useRef } from 'react';

function StreamingChat() {
    const [message, setMessage] = useState('');
    const [response, setResponse] = useState('');
    const [isStreaming, setIsStreaming] = useState(false);
    const eventSourceRef = useRef(null);
    
    const handleSubmit = async (e) => {
        e.preventDefault();
        if (!message.trim() || isStreaming) return;
        
        setResponse('');
        setIsStreaming(true);
        
        // Close any existing connection
        if (eventSourceRef.current) {
            eventSourceRef.current.close();
        }
        
        const eventSource = new EventSource(
            `/api/chatbot/stream?message=${encodeURIComponent(message)}`
        );
        eventSourceRef.current = eventSource;
        
        eventSource.onmessage = (event) => {
            if (event.data === '[DONE]') {
                eventSource.close();
                setIsStreaming(false);
                return;
            }
            
            try {
                const data = JSON.parse(event.data);
                if (data.chunk) {
                    setResponse(prev => prev + data.chunk);
                }
            } catch (e) {
                console.error('Parse error:', e);
            }
        };
        
        eventSource.onerror = () => {
            eventSource.close();
            setIsStreaming(false);
        };
    };
    
    useEffect(() => {
        return () => {
            if (eventSourceRef.current) {
                eventSourceRef.current.close();
            }
        };
    }, []);
    
    return (
        <div className="streaming-chat">
            <form onSubmit={handleSubmit}>
                <input
                    type="text"
                    value={message}
                    onChange={(e) => setMessage(e.target.value)}
                    placeholder="Ask something..."
                    disabled={isStreaming}
                />
                <button type="submit" disabled={isStreaming}>
                    {isStreaming ? 'Streaming...' : 'Send'}
                </button>
            </form>
            
            {response && (
                <div className="response">
                    <pre>{response}</pre>
                    {isStreaming && <span className="cursor">▋</span>}
                </div>
            )}
        </div>
    );
}

export default StreamingChat;

Vue 3 Component

<template>
  <div class="streaming-chat">
    <form @submit.prevent="handleSubmit">
      <input
        v-model="message"
        type="text"
        placeholder="Ask something..."
        :disabled="isStreaming"
      />
      <button type="submit" :disabled="isStreaming">
        {{ isStreaming ? 'Streaming...' : 'Send' }}
      </button>
    </form>
    
    <div v-if="response" class="response">
      <pre>{{ response }}</pre>
      <span v-if="isStreaming" class="cursor">▋</span>
    </div>
  </div>
</template>

<script setup>
import { ref, onUnmounted } from 'vue';

const message = ref('');
const response = ref('');
const isStreaming = ref(false);
let eventSource = null;

const handleSubmit = () => {
  if (!message.value.trim() || isStreaming.value) return;
  
  response.value = '';
  isStreaming.value = true;
  
  if (eventSource) {
    eventSource.close();
  }
  
  eventSource = new EventSource(
    `/api/chatbot/stream?message=${encodeURIComponent(message.value)}`
  );
  
  eventSource.onmessage = (event) => {
    if (event.data === '[DONE]') {
      eventSource.close();
      isStreaming.value = false;
      return;
    }
    
    try {
      const data = JSON.parse(event.data);
      if (data.chunk) {
        response.value += data.chunk;
      }
    } catch (e) {
      console.error('Parse error:', e);
    }
  };
  
  eventSource.onerror = () => {
    eventSource.close();
    isStreaming.value = false;
  };
};

onUnmounted(() => {
  if (eventSource) {
    eventSource.close();
  }
});
</script>

<style scoped>
.cursor {
  animation: blink 1s infinite;
}

@keyframes blink {
  0%, 50% { opacity: 1; }
  51%, 100% { opacity: 0; }
}
</style>

Angular Component

import { Component, OnDestroy } from '@angular/core';

@Component({
  selector: 'app-streaming-chat',
  template: `
    <div class="streaming-chat">
      <form (submit)="handleSubmit($event)">
        <input
          [(ngModel)]="message"
          type="text"
          placeholder="Ask something..."
          [disabled]="isStreaming"
          name="message"
        />
        <button type="submit" [disabled]="isStreaming">
          {{ isStreaming ? 'Streaming...' : 'Send' }}
        </button>
      </form>
      
      <div *ngIf="response" class="response">
        <pre>{{ response }}</pre>
        <span *ngIf="isStreaming" class="cursor">▋</span>
      </div>
    </div>
  `
})
export class StreamingChatComponent implements OnDestroy {
  message = '';
  response = '';
  isStreaming = false;
  private eventSource?: EventSource;
  
  handleSubmit(event: Event): void {
    event.preventDefault();
    
    if (!this.message.trim() || this.isStreaming) return;
    
    this.response = '';
    this.isStreaming = true;
    
    if (this.eventSource) {
      this.eventSource.close();
    }
    
    this.eventSource = new EventSource(
      `/api/chatbot/stream?message=${encodeURIComponent(this.message)}`
    );
    
    this.eventSource.onmessage = (event) => {
      if (event.data === '[DONE]') {
        this.eventSource?.close();
        this.isStreaming = false;
        return;
      }
      
      try {
        const data = JSON.parse(event.data);
        if (data.chunk) {
          this.response += data.chunk;
        }
      } catch (e) {
        console.error('Parse error:', e);
      }
    };
    
    this.eventSource.onerror = () => {
      this.eventSource?.close();
      this.isStreaming = false;
    };
  }
  
  ngOnDestroy(): void {
    if (this.eventSource) {
      this.eventSource.close();
    }
  }
}

Error Handling

Backend Error Handling

<?php
use Rumenx\PhpChatbot\Contracts\StreamableModelInterface;

try {
    // Check if model supports streaming
    if (!$model instanceof StreamableModelInterface) {
        throw new \RuntimeException('Model does not support streaming');
    }
    
    if (!$model->supportsStreaming()) {
        throw new \RuntimeException('Streaming not available for this model');
    }
    
    // Stream response
    foreach ($chatbot->askStream($message) as $chunk) {
        echo "data: " . json_encode(['chunk' => $chunk]) . "\n\n";
        flush();
    }
    
    echo "data: [DONE]\n\n";
    
} catch (\RuntimeException $e) {
    // Model doesn't support streaming - fallback
    $reply = $chatbot->ask($message);
    echo "data: " . json_encode(['chunk' => $reply]) . "\n\n";
    echo "data: [DONE]\n\n";
    
} catch (\Exception $e) {
    // General error
    echo "data: " . json_encode(['error' => $e->getMessage()]) . "\n\n";
}

flush();

Frontend Error Handling

const eventSource = new EventSource('/api/chatbot/stream?message=' + encodeURIComponent(message));

eventSource.onmessage = function(event) {
    if (event.data === '[DONE]') {
        eventSource.close();
        return;
    }
    
    try {
        const data = JSON.parse(event.data);
        
        if (data.error) {
            console.error('Stream error:', data.error);
            displayError(data.error);
            eventSource.close();
            return;
        }
        
        if (data.chunk) {
            appendToChat(data.chunk);
        }
    } catch (e) {
        console.error('Parse error:', e);
        eventSource.close();
    }
};

eventSource.onerror = function(error) {
    console.error('EventSource failed:', error);
    eventSource.close();
    displayError('Connection lost. Please try again.');
};

// Add timeout
setTimeout(() => {
    if (eventSource.readyState !== EventSource.CLOSED) {
        eventSource.close();
        displayError('Request timed out');
    }
}, 30000); // 30 seconds

Best Practices

1. Always Check Streaming Support

<?php
if ($model instanceof StreamableModelInterface && $model->supportsStreaming()) {
    // Use streaming
} else {
    // Use regular ask()
}

2. Disable Output Buffering

For PHP streaming to work properly:

<?php
// In your PHP configuration or script
@ini_set('output_buffering', 'off');
@ini_set('zlib.output_compression', false);

// Flush after each chunk
echo $chunk;
flush();

3. Set Proper Headers

Always set SSE headers:

<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
header('Connection: keep-alive');
header('X-Accel-Buffering: no'); // For nginx

4. Handle Connection Cleanup

Always close EventSource connections:

// React example
useEffect(() => {
    return () => {
        if (eventSourceRef.current) {
            eventSourceRef.current.close();
        }
    };
}, []);

5. Implement Timeouts

Prevent hanging connections:

const timeout = setTimeout(() => {
    eventSource.close();
    console.error('Stream timeout');
}, 60000); // 60 seconds

eventSource.onmessage = (event) => {
    clearTimeout(timeout); // Reset on each message
    // ... handle message
};

6. Rate Limiting

Implement rate limiting for streaming endpoints:

<?php
// Laravel example
Route::post('/api/chatbot/stream', [ChatbotController::class, 'stream'])
    ->middleware('throttle:10,1'); // 10 requests per minute

Troubleshooting

Streaming Not Working

Problem: No chunks are received on frontend.

Solutions:

  1. Check output buffering is disabled
  2. Verify headers are set correctly
  3. Check nginx configuration (disable buffering)
  4. Ensure flush() is called after each chunk
  5. Check browser console for errors

Chunks Arrive All at Once

Problem: All chunks arrive together at the end.

Solutions:

  1. Disable output buffering: @ini_set('output_buffering', 'off')
  2. Add to nginx config: proxy_buffering off;
  3. Ensure proper flushing

EventSource Connection Fails

Problem: EventSource shows error immediately.

Solutions:

  1. Check CORS headers if frontend is on different domain
  2. Verify endpoint URL is correct
  3. Check server logs for PHP errors
  4. Ensure Content-Type is text/event-stream

Memory Issues

Problem: High memory usage during streaming.

Solutions:

  1. Streaming uses less memory than buffering entire response
  2. If still high, check for memory leaks in your code
  3. Monitor with: echo "Memory: " . memory_get_usage() . "\n";

Nginx Configuration

If using nginx, add to your site config:

location /api/chatbot/stream {
    proxy_pass http://your-backend;
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header Connection '';
    proxy_http_version 1.1;
    chunked_transfer_encoding off;
}

Performance Considerations

  • Streaming uses similar total time but improves perceived performance
  • Connection overhead is slightly higher than regular requests
  • Memory efficient - chunks are processed and discarded
  • Works well with HTTP/2 for multiplexing multiple streams

Security Notes

  • Apply same rate limiting as regular endpoints
  • Validate and sanitize input before streaming
  • Implement authentication/authorization
  • Set reasonable timeouts
  • Monitor for abuse patterns

Next Steps

Clone this wiki locally