Description
Issue encountered
Evaluating models with transformers is very slow
Solution/Feature
Continuous batching for transformers models.
{{ message }}
No projects
No milestone
None yet
No branches or pull requests
Evaluating models with transformers is very slow
Continuous batching for transformers models.