Closed
Description
Description
Currently the user can limit the prompt input size and model output size only when using emulated data type.
Please separate the input & output sizes configuration from the type of the data.
Suggested Implementation
Add --data-max-input-tokens
and --max-output-tokens
parameters to the CLI.
If the data-type is file
or transformers
and the input size is larger than data_max_input_tokens
it should cut-off the input at the limit.
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
Done