Skip to content

Add VDB to directory structure in Rules.md #320

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bbelgodere merged 3 commits into from
Apr 9, 2026
+130 −92
Merged

Add VDB to directory structure in Rules.md #320

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4214ac6
Add VDB to directory structure in Rules.md
FileSystemGuy Apr 7, 2026
2e130c2
Add AiSEQ, DiskANN, and HNSW directories to rules
FileSystemGuy Apr 7, 2026
94dd456
Revise table of contents formatting in Rules.md
FileSystemGuy Apr 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
222 changes: 130 additions & 92 deletions Rules.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,41 +1,41 @@
# MLPerf™ Storage V2.0 Benchmark Validation Rules
——————————————————————————————————————————

- [MLPerf Storage V2.0 Benchmark Validation Rules](#mlperf-storage-v20-benchmark-validation-rules)
- [1. Introduction](#1-introduction)
- [2. Core/Common Rules](#2-core-common-rules)
- [2.1. Core/Common POSIX API Rules](#21-ccore-common-posix-api-rules)
- [2.2. Core/Common Object API Rules](#22-ccore-common-object-api-rules)
- [3. Validating the Training Options](#3-validating-the-training-options)
- [3.1. Training Sizing Options](#31-training-sizing-options)
- [3.2. Training Generation Options](#32-training-ganeration-options)
- [3.3. Training Run Options](#33-training-run-options)
- [3.4. Training Access Via POSIX API Options](#34-training-access-via-posix-api-options)
- [3.5. Training Access Via Object API Options](#35-training-access-via-object-API-options)
- [3.6. Training OPEN versus CLOSED Options](#36-training-open-versus-closed-options)
- [4. Validating the Checkpointing Options](#4-validating-the-checkpointing-options)
- [4.1. Checkpointing Sizing Options](#41-checkpointing-sizing-options)
- [4.2. Checkpointing Generation Options](#42-checkpointing-generation-options)
- [4.3. Checkpointing Run Options](#43-checkpointing-run-options)
- [4.4. Checkpointing Access Via POSIX API Options](#44-checkpointing-access-via-posix-api-options)
- [4.5. Checkpointing Access Via Object API Options](#45-checkpointing-access-via-object-API-options)
- [4.6. Checkpointing OPEN versus CLOSED Options](#46-checkpointing-open-versus-closed-options)
- [4.7. Storage System Must Be Simultaneously R/W or Remappable](#47-storage-system-must-be-simultaneously-rw-or-remappable)
- [5. Validating the VDB Options](#5-validating-the-vdb-options)
- [5.1. VDB Sizing Options](#51-vdb-sizing-options)
- [5.2. VDB Generation Options](#52-vdb-generation-options)
- [5.3. VDB Run Options](#53-vdb-run-options)
- [5.4. VDB Access Via POSIX API Options](#54-vdb-access-via-posix-api-options)
- [5.5. VDB Via Object API Options](#55-vdb-access-via-object-API-options)
- [5.6. VDB OPEN versus CLOSED Options](#56-vdb-open-versus-closed-options)
- [6. Validating the KVCache Options](#6-validating-the-kvcache-options)
- [6.1. KVCache Sizing Options](#61-kvcache-sizing-options)
- [6.2. KVCache Generation Options](#62-kvcache-generation-options)
- [6.3. KVCache Run Options](#63-kvcache-run-options)
- [6.4. KVCache Access Via POSIX API Options](#64-kvcache-access-via-posix-api-options)
- [6.5. KVCache Access Via Object API Options](#65-kvcache-access-via-object-API-options)
- [6.6. KVCache OPEN versus CLOSED Options](#66-kvcache-open-versus-closed-options)

# Table of Contents

* [1. Introduction](#1-introduction)
* [2. Core/Common Rules](#2-corecommon-rules)
* [2.1. Core/Common POSIX API Rules](#21-corecommon-posix-api-rules)
* [2.2. Core/Common Object API Rules](#22-corecommon-object-api-rules)
* [3. Validating the Training Options](#3-validating-the-training-options)
* [3.1. Training Sizing Options](#31-training-sizing-options)
* [3.2. Training Generation Options](#32-training-generation-options)
* [3.3. Training Run Options](#33-training-run-options)
* [3.4. Training Access Via POSIX API Options](#34-training-access-via-posix-api-options)
* [3.5. Training Access Via Object API Options](#35-training-access-via-object-api-options)
* [3.6. Training OPEN versus CLOSED Options](#36-training-open-versus-closed-options)
* [4. Validating the Checkpointing Options](#4-validating-the-checkpointing-options)
* [4.1. Checkpointing Sizing Options](#41-checkpointing-sizing-options)
* [4.2. Checkpointing Generation Options](#42-checkpointing-generation-options)
* [4.3. Checkpointing Run Options](#43-checkpointing-run-options)
* [4.4. Checkpointing Access Via POSIX API Options](#44-checkpointing-access-via-posix-api-options)
* [4.5. Checkpointing Access Via Object API Options](#45-checkpointing-access-via-object-api-options)
* [4.6. Checkpointing OPEN versus CLOSED Options](#46-checkpointing-open-versus-closed-options)
* [4.7. Storage System Must Be Simultaneously R/W or Remappable](#47-storage-system-must-be-simultaneously-rw-or-remappable)
* [5. Validating the VDB Options](#5-validating-the-vdb-options)
* [5.1. VDB Sizing Options](#51-vdb-sizing-options)
* [5.2. VDB Generation Options](#52-vdb-generation-options)
* [5.3. VDB Run Options](#53-vdb-run-options)
* [5.4. VDB Access Via POSIX API Options](#54-vdb-access-via-posix-api-options)
* [5.5. VDB Access Via Object API Options](#55-vdb-access-via-object-api-options)
* [5.6. VDB OPEN versus CLOSED Options](#56-vdb-open-versus-closed-options)
* [6. Validating the KVCache Options](#6-validating-the-kvcache-options)
* [6.1. KVCache Sizing Options](#61-kvcache-sizing-options)
* [6.2. KVCache Generation Options](#62-kvcache-generation-options)
* [6.3. KVCache Run Options](#63-kvcache-run-options)
* [6.4. KVCache Access Via POSIX API Options](#64-kvcache-access-via-posix-api-options)
* [6.5. KVCache Access Via Object API Options](#65-kvcache-access-via-object-api-options)
* [6.6. KVCache OPEN versus CLOSED Options](#66-kvcache-open-versus-closed-options)
# 1. Introduction

These are the requirements for the *submission validation checker* for version 2.0 of the MLPerf™ Storage benchmark,
Expand Down Expand Up @@ -156,35 +156,54 @@ root_folder (or any name you prefer)
│ │ │ ... (5x Runs per Emulated Accelerator Type)
│ │ │ └── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ └── checkpointing
│ │ ├── llama3-8b
│ │ │ ├──results.json
│ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ └── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ├── llama3-70b
│ │ │ ├──results.json
│ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ └── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ├── llama3-405b
│ │ │ ├──results.json
│ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ └── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ └── llama3-1t
│ │ ├──results.json
│ │ ├── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ ├── checkpointing
│ │ │ ├── llama3-8b
│ │ │ │ ├──results.json
│ │ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ │ └── dlio_config
│ │ │ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ │ └── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ │ ├── llama3-70b
│ │ │ │ ├──results.json
│ │ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ │ └── dlio_config
│ │ │ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ │ └── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ │ ├── llama3-405b
│ │ │ │ ├──results.json
│ │ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ │ └── dlio_config
│ │ │ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ │ └── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ │ └── llama3-1t
│ │ │ ├──results.json
│ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ └── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ └── vdb_bench
| | ├── AiSEQ
Copy link
Copy Markdown
Contributor

@idevasena idevasena Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Typo. AiSEQ --> AiSAQ

Copy link
Copy Markdown
Contributor Author

@FileSystemGuy FileSystemGuy Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll pick this up in the next round of changes...

│ │ | ├── YYYYMMDD_HHmmss
│ │ | │ └── summary.json
│ │ | ... (5x Runs total)
│ │ | └── YYYYMMDD_HHmmss
│ │ | └── summary.json
| | ├── DiskANN
│ │ | ├── YYYYMMDD_HHmmss
│ │ | │ └── summary.json
│ │ | ... (5x Runs total)
│ │ | └── YYYYMMDD_HHmmss
│ │ | └── summary.json
| | └── HNSW
│ │ ├── YYYYMMDD_HHmmss
│ │ │ └── summary.json
│ │ ... (5x Runs total)
│ │ └── YYYYMMDD_HHmmss
│ │ └── dlio_config
│ │ └── summary.json
│ └── systems
│ ├──system-name-1.yaml
│ ├──system-name-1.pdf
Expand Down Expand Up @@ -230,35 +249,54 @@ root_folder (or any name you prefer)
│ │ ... (5x Runs per Emulated Accelerator Type)
│ │ └── YYYYMMDD_HHmmss
│ │ └── dlio_config
│ └── checkpointing
│ ├── llama3-8b
│ | ├──results.json
│ │ ├── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ └── YYYYMMDD_HHmmss
│ │ └── dlio_config
│ ├── llama3-70b
│ | ├──results.json
│ │ ├── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ └── YYYYMMDD_HHmmss
│ │ └── dlio_config
│ ├── llama3-405b
│ | ├──results.json
│ │ ├── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ └── YYYYMMDD_HHmmss
│ │ └── dlio_config
│ └── llama3-1t
│ ├──results.json
│ ├── YYYYMMDD_HHmmss
│ │ └── dlio_config
│ ... (10x Runs for Read and Write. May be combined in a single run)
│ └── YYYYMMDD_HHmmss
│ └── dlio_config
│ ├── checkpointing
│ │ ├── llama3-8b
│ │ │ ├──results.json
│ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ └── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ├── llama3-70b
│ │ │ ├──results.json
│ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ └── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ├── llama3-405b
│ │ │ ├──results.json
│ │ │ ├── YYYYMMDD_HHmmss
│ │ │ │ └── dlio_config
│ │ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ │ └── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ └── llama3-1t
│ │ ├──results.json
│ │ ├── YYYYMMDD_HHmmss
│ │ │ └── dlio_config
│ │ ... (10x Runs for Read and Write. May be combined in a single run)
│ │ └── YYYYMMDD_HHmmss
│ │ └── dlio_config
│ └── vdb_bench
| ├── AiSEQ
│ | ├── YYYYMMDD_HHmmss
│ | │ └── summary.json
│ | ... (5x Runs total)
│ | └── YYYYMMDD_HHmmss
│ | └── summary.json
| ├── DiskANN
│ | ├── YYYYMMDD_HHmmss
│ | │ └── summary.json
│ | ... (5x Runs total)
│ | └── YYYYMMDD_HHmmss
│ | └── summary.json
| └── HNSW
│ ├── YYYYMMDD_HHmmss
│ │ └── summary.json
│ ... (5x Runs total)
│ └── YYYYMMDD_HHmmss
│ └── summary.json
└── systems
├──system-name-1.yaml
├──system-name-1.pdf
Expand Down
Loading