Skip to content

Commit 752fbf0

Browse files
authored
[Doc] Introduce the validation tool, benchgc (#179)
1 parent 67b75ca commit 752fbf0

File tree

1 file changed

+39
-0
lines changed

1 file changed

+39
-0
lines changed

docs/benchgc_overview.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# BenchGC Overview
2+
3+
BenchGC is a testing tool for the Graph Compiler project, focusing on tuning performance and verifying the correctness of the Graph Compiler. The benchmark and validation work will be supported on both CPU and GPU.
4+
5+
BenchGC will also be part of the CI test and checking the regular correctness check for each PR.
6+
7+
### What can BenchGC provide?
8+
* Working modes
9+
- Correctness
10+
- Performance
11+
- Tuning
12+
* Validation scope
13+
- Single MLIR linalg op
14+
- Predefined pattern (such as MLP)
15+
- MLIR Module provided by user
16+
* Data filling strategy
17+
- Benchdnn filling strategy (mainly for single op validation)
18+
- Common distribution filling
19+
- Load from pytorch dump
20+
* Result comparison strategy
21+
- Benchdnn compare strategy (mainly for single op validation)
22+
- User defined threshold
23+
24+
### BenchGC workflow
25+
```mermaid
26+
flowchart TB
27+
SINGLE_OP_TEST["Single Op Test"] -- dialect, op, shape, dtype, attr --> MLIR_MODULE["MLIR Module"]
28+
USER_MLIR_TEST["User MLIR Test"] -- load a MLIR file --> MLIR_MODULE["MLIR Module"]
29+
PATTERN_TEST["Pattern Test"] -- pattern specified attr, shape, dtype --> MLIR_MODULE["MLIR Module"]
30+
MLIR_MODULE["MLIR Module"] -- correctness --> REF_MLIR_RUNNER["Reference MLIR Runner"]
31+
MLIR_MODULE["MLIR Module"] --> GRAPH_COMPILER_LIBRARY["Graph Compiler Library"]
32+
REF_MLIR_RUNNER["Reference MLIR Runner"] -- pytorch --> COMPARISON{"Compare"}
33+
GRAPH_COMPILER_LIBRARY["Graph Compiler Library"] -- correctness --> COMPARISON{"Compare"}
34+
GRAPH_COMPILER_LIBRARY["Graph Compiler Library"] -- performance --> PERF_RESULT["Performance Result"]
35+
COMPARISON{"Compare"} --> PASS["Pass"]
36+
COMPARISON{"Compare"} --> FAIL["Fail"]
37+
```
38+
### Acknowledgements
39+
This work is inspired by the [benchdnn](https://github.com/oneapi-src/oneDNN/tree/main/tests/benchdnn) component from [oneDNN](https://github.com/oneapi-src/oneDNN/) project developed by Intel.

0 commit comments

Comments
 (0)