Description
Feature Request
Is your feature request related to a problem? Please describe.
Currently, the system is limited to using TiktokenModel for token trimming, which restricts compatibility with non-OpenAI models. This limitation may hinder the integration of diverse models that could enhance performance and scalability.
Describe the solution you'd like
Implement a flexible token trimming mechanism that supports a variety of models beyond TiktokenModel. This could involve abstracting the token trimming logic to accommodate different model architectures and tokenization strategies.
Describe alternatives you've considered
Continuing with the current model-specific approach, but this would limit the flexibility and potential for optimization across different models.
Additional context
Expanding support for non-OpenAI models will improve the system's adaptability and allow for better optimization of token trimming processes. This aligns with the goal of enhancing algorithm efficiency and scalability.
Related Issues
None at the moment, but tracking this enhancement will facilitate discussions around implementation strategies.
Activity