Skip to content

Pareto clarification #84

@arekay-nv

Description

@arekay-nv

The detailed definition of pareto submissions needs to be described. This includes the following:

  • What models are going to be used by each submitters
  • What dataset is going to be used for each model/submitter
  • How are points defined (concurrency/latency/throughput) and what points are needed for submissions

Models

Current candidates are:

  • Deepseek
  • gpt-oss
  • llama2-70b

Dataset

Ideally we would like dataset not currently used by MLPerf to avoid comparability.

Pareto points

Metadata

Metadata

Labels

area: config-cliConfig schema, CLI commands, YAMLpriority: ShowStopperDrop everything — critical blocker, all hands on deck

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions