Skip to content

Automatic performance stabilizization - so no need to set max-requests or max-seconds #101

@rgreenberg1

Description

@rgreenberg1

Description
It would be great if we could enable automatic performance stabilization so they user would not have to estimate max-requests or max-seconds and potentially still not find the point where performance plateaus and stabilizes. it would be great to add in some automatic logic for - Something along the lines of ensuring a combination across requests and time.

Acceptance Criteria
Add in automated logic to ensure GuideLLM is able to find the performance stabilization point when running a benchmark.

Proposal:
Can set max-seconds to accept an auto flag, and we can have it measure when the benchmarks have stabilized, then will stop sending new requests and times out/stops benchmarking.

Metadata

Metadata

Assignees

Labels

analysisanalysis workstream - relating to adding in features to do analysis on extrapolating resultsautomationautomation workstreampriority-high

Projects

Status

Ready

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions