fix: non default temperature check

[lovets18]

Change
Added check for models o3-mini, o4-mini, o3

Reason
Models o3-mini, o4-mini, o3 has temperature, but they don't support setting it to non default value, so I encountered error:

BadRequestError: Error code: 400 - {
    'error’: {
        'message': "Unsupported value: 'temperature' does not support 1E-8 with this model. Only the default (1) value is supported.", 
        'type': 'invalid_request_error’, 
        'param': 'temperature’, 
        'code': 'unsupported_value’
    }
}

[greptile-apps]

Greptile Summary

This PR adds temperature restriction handling for specific OpenAI models (o3-mini, o4-mini, o3) that only support default temperature values. The change introduces a hardcoded list MODELS_NOT_SUPPORT_TEMP containing these model names and modifies the LangchainLLMWrapper class to check against this list before setting temperature parameters.

The fix addresses a specific API limitation where newer OpenAI models throw a BadRequestError when temperature is set to non-default values (like the 1e-8 that Ragas uses for deterministic outputs). The implementation adds conditional checks in both synchronous and asynchronous text generation methods (generate_text and agenerate_text) that only set the temperature if the model is not in the restricted list.

This change fits into the broader Ragas architecture by extending the existing LLM wrapper functionality to handle model-specific API constraints. The fix maintains backward compatibility while enabling support for these newer OpenAI models without requiring users to modify their code.

Confidence score: 3/5

• This PR addresses a real API limitation but has implementation concerns that could lead to inconsistent behavior
• The score reflects potential issues with the conditional logic structure and maintainability of hardcoded model lists
• Files that need more attention: ragas/src/ragas/llms/base.py - the temperature restriction logic should be reviewed for consistency across different LLM implementations

The main concerns are:

1. Inconsistent behavior: The temperature restriction is only applied when both temperature and model_name attributes exist, which could create different behavior across LLM implementations
2. Maintainability: The hardcoded model list may become outdated as OpenAI releases new models with similar restrictions
3. Missing coverage: The fix only applies to LangchainLLMWrapper but other LLM implementations in the codebase might face the same issue

_{1 file reviewed, 1 comment}

_{Edit Code Review Bot Settings | Greptile}