New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

128K上下文的，10万字超长文本，导致过滤器持续吃CPU #73

Open

ws-liusw opened this issue Aug 19, 2024 · 1 comment

ws-liusw commented Aug 19, 2024

128K上下文的，10万字超长文本，导致过滤器线程持续吃CPU，挤占资源

Owner

houbb commented Nov 11, 2024

可以考虑在业务代码中加一个判断，如果单个文本特别长，是否可以用简单的脱敏策略处理？（比如移除 URL/EMAIL 等待正则的判断）
如果业务必须，那就加机器。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment