Skip to content

Rethink retry logic for LLM Providers #305

@Munsio

Description

@Munsio

This image shows the uptime of the RWKV v5 World 3B model:
image

This model is so long down that the retry logic we are currently using does not really work, but waiting for the model to be available again would only stretch out the evaluation runs artificially

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpostponedThis issue/PR is postponed until there is a very good reason (e.g. $$$) to implement it.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions