Skip to content

Record partially completed request metrics #77

Closed
@sjmonson

Description

@sjmonson

At the end of a test, guidellm cancels all pending requests. This causes the reported throughput to be lower depending on the portion of the overall test that the canceled requests cover. Instead we should record some kind of partial completions for the tokens completed within the deadline.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions