Fix Ollama client streaming issue with stream=True

[devdattatalele]

Summary

Fixes issue #299 where the Ollama client fails with 'generator' object has no attribute 'raw_response' error when using stream=True.

Correct Implementation

• OllamaClient consistently returns GeneratorOutput for both streaming and non-streaming
• raw_response contains the streaming generator (following established contract)
• data remains None until final output is processed by Generator component
• Maintains interface consistency across all model clients
• Preserves polymorphism - all clients return the same type

Previous Approach (Fixed)

The initial implementation incorrectly returned raw generators directly, breaking:

• Interface consistency across model clients
• Generator component expectations
• SOLID principles (Single Responsibility, Open/Closed)

Technical Changes

1. OllamaClient (adalflow/components/model_client/ollama_client.py)

# CORRECT: Consistent GeneratorOutput return
if isinstance(completion, GeneratorType):
    return GeneratorOutput(
        data=None,                    # Final output processed later
        raw_response=completion,      # Streaming iterator
        api_response=completion       # Original response
    )

2. Generator Core (adalflow/core/generator.py)

• Removed incorrect type checking logic that was specific to OllamaClient
• Restored original error handling that works for all model clients
• Maintains single responsibility principle

3. Test Updates (tests/test_ollama_client.py)

Updated to follow proper streaming contract:

• parsed.raw_response → streaming iterator
• parsed.data → None (until consumed)
• Maintains GeneratorOutput consistency

Test Results

# All tests passing
pytest tests/test_ollama_client.py -v
======================== 10 passed ========================

#299

[liyin2015]

we almost never change the generator output. Raw_response is for streaming, data is for final parsed result

[liyin2015]

check anthropic client on how to handle the streaming. Eventually the standard is to convert to openai's responses api standard.

[liyin2015]

try this code,

from adalflow.components.model_client.ollama_client import OllamaClient
from adalflow.core import Generator

stream_generator = Generator(
    model_client=OllamaClient(host="http://localhost:11434"),
    model_kwargs={
        "model": "gpt-oss:20b",
        "stream": True,  # Enable streaming
    }
)

async def test_ollama_streaming():
    # async call with streaming
    output = await stream_generator.acall(prompt_kwargs={"input_str": "Why is the sky blue?"})

    async for chunk in output.raw_response:
        print(chunk["message"]["content"], end='', flush=True)


if __name__ == "__main__":
    import asyncio
    asyncio.run(test_ollama_streaming())

It works for streaming.

[devdattatalele]

@liyin2015 I've updated the implementation based on your feedback. You mentioned that raw_response should stay for streaming and to check the Anthropic client for reference. I tested it with your code example it works perfectly!

[liyin2015]

check the comments

[liyin2015]

this pr does not do much. and we cant force the data to None either. It is supposed to be the final complete output data, which should be handled in ollama_client, where u have to collect all stream and save the complete one in this field. you can see example in https://github.com/SylphAI-Inc/AdalFlow/blob/main/adalflow/adalflow/components/model_client/anthropic_client.py

[liyin2015]

keep the generator not changed at all