Skip to content

Conversation

fangyinc
Copy link
Collaborator

@fangyinc fangyinc commented Mar 7, 2025

Description

  1. DB-GPT API supports reasoning content
  2. Chat Data & Chat Dashboard supports stream output
  3. Call DB-GPT API by OpenAI SDK

How Has This Been Tested?

Call DB-GPT API by OpenAI SDK

from openai import OpenAI

DBGPT_API_KEY = "dbgpt"

client = OpenAI(
    api_key=DBGPT_API_KEY,
    base_url="http://localhost:5670/api/v2",
)

messages = [
    {
        "role": "user",
        "content": "Hello, how are you?",
    },
]

has_thinking = False
reasoning_content = ""
for chunk in client.chat.completions.create(
    model="deepseek-chat",
    messages=messages,
    extra_body={
        "chat_mode": "chat_normal",
    },
    stream=True,
    max_tokens=4096,
):
    delta_content = chunk.choices[0].delta.content
    if hasattr(chunk.choices[0].delta, "reasoning_content"):
        reasoning_content = chunk.choices[0].delta.reasoning_content
    if reasoning_content:
        if not has_thinking:
            print("<thinking>", flush=True)
        print(reasoning_content, end="", flush=True)
        has_thinking = True
    if delta_content:
        if has_thinking:
            print("</thinking>", flush=True)
        print(delta_content, end="", flush=True)
        has_thinking = False

More detail see examples/client/client_openai_chat.py

Snapshots:

Include snapshots for easier review.

Checklist:

  • My code follows the style guidelines of this project
  • I have already rebased the commits and make the commit message conform to the project standard.
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • Any dependent changes have been merged and published in downstream modules

@github-actions github-actions bot added enhancement New feature or request model Module: model labels Mar 7, 2025
Copy link
Collaborator

@Aries-ckt Aries-ckt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@yyhhyyyyyy yyhhyyyyyy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yyhhyyyyyy yyhhyyyyyy merged commit 4e993a2 into eosphoros-ai:main Mar 7, 2025
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request model Module: model
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants