Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: unnecessary auto-increment calculations in the tokens statistics of the chat model #2968

Closed
1 task done
yqwang96 opened this issue Oct 22, 2024 · 0 comments
Closed
1 task done
Labels
bug Something isn't working

Comments

@yqwang96
Copy link
Contributor

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Branch name

main

Commit ID

aaaa

Other environment information

No response

Actual behavior

In the chat_streamly method of the Base class in \rag\llm\chat_model.py, there is an unnecessary auto-increment in the calculation of total_tokens;

Expected behavior

No response

Steps to reproduce

total_tokens += 1
                if not hasattr(resp, "usage") or not resp.usage:
                    total_tokens = (
                                total_tokens
                                + num_tokens_from_string(resp.choices[0].delta.content)
                        )
                elif isinstance(resp.usage, dict):
                    total_tokens = resp.usage.get("total_tokens", total_tokens)
                else:
                    total_tokens = resp.usage.total_tokens

Additional information

No response

@yqwang96 yqwang96 added the bug Something isn't working label Oct 22, 2024
KevinHuSh added a commit that referenced this issue Oct 22, 2024
…cs of the chat model (#2969)

### What problem does this PR solve?

the details is shown in
#2968

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant