Skip to content

Commit

Permalink
Python: New AI Connector abstract methods (microsoft#8526)
Browse files Browse the repository at this point in the history
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This PR implements:
https://github.com/microsoft/semantic-kernel/blob/main/docs/decisions/0052-python-ai-connector-new-abstract-methods.md
(PR: microsoft#8430).

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
1. Add abstract methods `_inner_get_chat_message_content` and
`_inner_get_streaming_chat_message_content` to
`ChatCompletionClientBase`.
2. Implement the abstractions in all chat completion connectors.
3. Add abstract methods `_inner_get_text_contents` and
`_inner_get_streaming_text_contents` to `TextCompletionClientBase`.
4. Implement the abstractions in all text completion connectors.
5. Remove text completion APIs from `OllamaChatCompletion` (breaking
changes).
> No breaking changes on other connectors except Ollama, and public APIs
stay the same.


### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
  • Loading branch information
TaoChenOSU authored Sep 9, 2024
1 parent 24294bf commit bd34da9
Show file tree
Hide file tree
Showing 26 changed files with 912 additions and 1,422 deletions.
12 changes: 6 additions & 6 deletions docs/decisions/0052-python-ai-connector-new-abstract-methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,24 +30,24 @@ Auto function invocation can cause a side effect where a single call to get_chat

### Two new abstract methods

> Revision: In order to not break existing customers who have implemented their own AI connectors, these two methods are not decorated with the `@abstractmethod` decorator, but instead throw an exception if they are not implemented in the built-in AI connectors.
```python
@abstractmethod
async def _send_chat_request(
async def _inner_get_chat_message_content(
self,
chat_history: ChatHistory,
settings: PromptExecutionSettings
) -> list[ChatMessageContent]:
pass
raise NotImplementedError
```

```python
@abstractmethod
async def _send_streaming_chat_request(
async def _inner_get_streaming_chat_message_content(
self,
chat_history: ChatHistory,
settings: PromptExecutionSettings
) -> AsyncGenerator[list[StreamingChatMessageContent], Any]:
pass
raise NotImplementedError
```

### A new `ClassVar[bool]` variable in `ChatCompletionClientBase` to indicate whether a connector supports function calling
Expand Down
3 changes: 2 additions & 1 deletion python/.cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@
"contoso",
"opentelemetry",
"SEMANTICKERNEL",
"OTEL"
"OTEL",
"mistralai"
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ class AnthropicChatCompletion(ChatCompletionClientBase):
"""Antropic ChatCompletion class."""

MODEL_PROVIDER_NAME: ClassVar[str] = "anthropic"
SUPPORTS_FUNCTION_CALLING: ClassVar[bool] = False

async_client: AsyncAnthropic

Expand Down Expand Up @@ -103,31 +104,25 @@ def __init__(
ai_model_id=anthropic_settings.chat_model_id,
)

# region Overriding base class methods

# Override from AIServiceClientBase
@override
def get_prompt_execution_settings_class(self) -> type["PromptExecutionSettings"]:
return AnthropicChatPromptExecutionSettings

@override
@trace_chat_completion(MODEL_PROVIDER_NAME)
async def get_chat_message_contents(
async def _inner_get_chat_message_contents(
self,
chat_history: "ChatHistory",
settings: "PromptExecutionSettings",
**kwargs: Any,
) -> list["ChatMessageContent"]:
"""Executes a chat completion request and returns the result.
Args:
chat_history: The chat history to use for the chat completion.
settings: The settings to use for the chat completion request.
kwargs: The optional arguments.
Returns:
The completion result(s).
"""
if not isinstance(settings, AnthropicChatPromptExecutionSettings):
settings = self.get_prompt_execution_settings_from_settings(settings)
assert isinstance(settings, AnthropicChatPromptExecutionSettings) # nosec

if not settings.ai_model_id:
settings.ai_model_id = self.ai_model_id

settings.ai_model_id = settings.ai_model_id or self.ai_model_id
settings.messages = self._prepare_chat_history_for_request(chat_history)
try:
response = await self.async_client.messages.create(**settings.prepare_settings_dict())
Expand All @@ -146,29 +141,17 @@ async def get_chat_message_contents(
self._create_chat_message_content(response, content_block, metadata) for content_block in response.content
]

async def get_streaming_chat_message_contents(
@override
async def _inner_get_streaming_chat_message_contents(
self,
chat_history: ChatHistory,
settings: PromptExecutionSettings,
**kwargs: Any,
) -> AsyncGenerator[list[StreamingChatMessageContent], Any]:
"""Executes a streaming chat completion request and returns the result.
Args:
chat_history: The chat history to use for the chat completion.
settings: The settings to use for the chat completion request.
kwargs: The optional arguments.
Yields:
A stream of StreamingChatMessageContent.
"""
chat_history: "ChatHistory",
settings: "PromptExecutionSettings",
) -> AsyncGenerator[list["StreamingChatMessageContent"], Any]:
if not isinstance(settings, AnthropicChatPromptExecutionSettings):
settings = self.get_prompt_execution_settings_from_settings(settings)
assert isinstance(settings, AnthropicChatPromptExecutionSettings) # nosec

if not settings.ai_model_id:
settings.ai_model_id = self.ai_model_id

settings.ai_model_id = settings.ai_model_id or self.ai_model_id
settings.messages = self._prepare_chat_history_for_request(chat_history)
try:
async with self.async_client.messages.stream(**settings.prepare_settings_dict()) as stream:
Expand All @@ -189,13 +172,14 @@ async def get_streaming_chat_message_contents(
]
elif isinstance(stream_event, ContentBlockStopEvent):
content_block_idx += 1

except Exception as ex:
raise ServiceResponseException(
f"{type(self)} service failed to complete the request",
ex,
) from ex

# endregion

def _create_chat_message_content(
self, response: Message, content: TextBlock, response_metadata: dict[str, Any]
) -> "ChatMessageContent":
Expand Down Expand Up @@ -249,7 +233,3 @@ def _create_streaming_chat_message_content(
finish_reason=finish_reason,
items=items,
)

def get_prompt_execution_settings_class(self) -> "type[AnthropicChatPromptExecutionSettings]":
"""Create a request settings object."""
return AnthropicChatPromptExecutionSettings
Loading

0 comments on commit bd34da9

Please sign in to comment.