You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: implement object pooling for PluginPipeline to improve performance (#146)
# Implement Object Pooling for PluginPipeline
This PR adds object pooling for `PluginPipeline` objects to improve performance by reducing memory allocations and garbage collection overhead. The implementation:
1. Adds a new `pluginPipelinePool` to the `Bifrost` struct
2. Implements methods to get, reset, and release pipeline objects
3. Prewarms the pool during initialization
4. Refactors the request handling code to use pooled pipelines
Additionally, this PR:
- Extracts common validation logic into a `validateRequest` helper function
- Moves utility functions like `providerRequiresKey` and `calculateBackoff` to the utils.go file
- Consolidates the try* methods into a single `tryRequest` function
- Simplifies the request handling flow
These changes should reduce memory pressure and improve performance for high-throughput scenarios.
0 commit comments