Official LangChain compatibility layer for LLM7 API services.
pip install langchain-llm7- 🚀 Native integration with LangChain's BaseChatModel interface
- ⚡ Support for both streaming and non-streaming responses
- 🔧 Customizable model parameters (temperature, max_tokens, stop sequences)
- 📊 Token usage metadata tracking
- 🛠 Robust error handling and retry mechanisms
from langchain_llm7 import ChatLLM7
from langchain_core.messages import HumanMessage
# Initialize with default parameters
llm = ChatLLM7()
# Basic invocation
response = llm.invoke([HumanMessage(content="Hello!")])
print(response.content)# Enable streaming
llm = ChatLLM7(streaming=True)
for chunk in llm.stream([HumanMessage(content="Tell me about quantum computing")]):
print(chunk.content, end="", flush=True)# Custom model configuration
llm = ChatLLM7(
model="llama-3.3-70b-instruct-fp8-fast",
temperature=0.7,
max_tokens=500,
stop=["\n", "Observation:"],
timeout=45
)| Parameter | Description | Default |
|---|---|---|
model |
Model version to use | "gpt-4o-mini-2024-07-18" |
base_url |
API endpoint URL | "https://api.llm7.io/v1" |
temperature |
Sampling temperature (0.0-2.0) | 1.0 |
max_tokens |
Maximum number of tokens to generate | None |
timeout |
Request timeout in seconds | 120 |
stop |
Stop sequences for response generation | None |
streaming |
Enable streaming response mode | False |
The library provides detailed error messages for:
- API communication failures
- Invalid message formats
- Unsupported message types
- Response parsing errors
try:
llm.invoke([{"invalid": "message"}])
except ValueError as e:
print(f"Error: {e}")To run the test suite:
pip install pytest
pytest tests/For complete documentation see:
Contributions are welcome! Please open an issue or submit a PR:
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Eugene Evstafev