Commit 7cc4bf0
committed
Fix PR, including logging, copyrights, and testing
Improve logging: don't log load mode when using request-intervals
When using request-intervals, the load pattern is user-specified
and fixed (not being varied by Model Analyzer), so we shouldn't
log it. This follows MA's principle of only logging parameters
that MA is actively sweeping/varying.
Output:
- Before: 'Profiling model_config_11: request-intervals=./intervals.txt'
- After: 'Profiling model_config_11' (or with batch size if > 1)
Fix test expectations after changing batch-size default to 1
Updated test_perf_analyzer.py to expect -f test_model-results.csv
and -b 1 in CLI strings, matching the changes from:
- Setting default batch-size to 1 in PerfAnalyzerConfig.__init__
- Restoring latency-report-file in setUp()
Three tests were failing because expected strings didn't include
these parameters:
- test_perf_analyzer_boolean_args
- test_perf_analyzer_additive_args
- test_perf_analyzer_ssl_args
Fix tests
Remove broken test_run and fix test_get_cmd_single_model
Changes:
1. Removed test_run method (lines 240-535)
- Doesn't exist on main
- Integration test requiring actual Triton/perf_analyzer binaries
- Has incomplete mocking, never passed
- Not related to request-intervals feature
2. Fixed test_get_cmd_single_model expected command
- Added -f test_model-results.csv to expected output
- This matches the actual output after we restored
latency-report-file in setUp()
Fix test_run by adding missing _verify_output_files_exist mock
The test_run method was failing because it was trying to actually
execute perf_analyzer and verify files exist. Added the missing
patch.object() mock for _verify_output_files_exist that returns True,
which was present in the original test_pa_csv_output but missing when
it was refactored into test_run.
Applied the mock to all 16 patch blocks in the test that call
perf_analyzer.run().
All 13 tests in test_perf_analyzer.py now pass.
Restore test_pa_llm_csv_output from main to avoid regression
The test_pa_llm_csv_output test exists on main and tests LLM metrics
parsing functionality. It was accidentally removed in the original PR's
"fix unit tests" commit (b21f41d) when the author consolidated tests
into test_run.
Since this is a feature branch, we must not introduce regressions from
main. Restored the test from main (lines 502-703) along with all 19
required LLM metric imports.
All 14 tests now pass.
Clean up comments
Replace test_run with test_pa_csv_output from main
The test_run test was created in the original PR to replace
test_pa_csv_output, but since main has test_pa_csv_output (not
test_run), we need to use main's version to avoid divergence.
Both test_run and test_pa_csv_output test PA CSV parsing, but
test_pa_csv_output is the canonical version on main. Replaced
test_run (lines 259-588) with test_pa_csv_output from main
(lines 271-501).
Our branch now matches main's test structure:
- test_pa_csv_output (general PA CSV parsing)
- test_pa_llm_csv_output (LLM-specific CSV parsing)
All 14 tests pass.
Add unit tests for request-intervals feature
Added 4 unit tests to verify the request-intervals feature works correctly:
1. test_request_intervals_in_inference_load_args
- Verifies request-intervals is in the inference_load_args list
- Confirms all 3 load args (concurrency, request-rate, request-intervals) are present
2. test_request_intervals_preserved_in_config
- Tests that request-intervals value is preserved when set in PerfAnalyzerConfig
3. test_request_intervals_in_cli_string
- Verifies request-intervals appears correctly in the CLI string
- Format: --request-intervals=./intervals.txt
4. test_request_intervals_mutually_exclusive
- Tests that when request-intervals is set, other load parameters
(concurrency-range, request-rate-range) do NOT appear in CLI string
- This is the core feature: preventing MA from overriding user's request-intervals
All 18 tests pass (14 existing + 4 new).1 parent 7b11b95 commit 7cc4bf0
File tree
12 files changed
+347
-291
lines changed- model_analyzer
- config/generate
- perf_analyzer
- plots
- record
- result
- tests
12 files changed
+347
-291
lines changedLines changed: 2 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
Lines changed: 2 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
Lines changed: 2 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
Lines changed: 2 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
785 | 773 | | |
786 | 774 | | |
787 | 775 | | |
788 | | - | |
| 776 | + | |
789 | 777 | | |
790 | 778 | | |
791 | | - | |
792 | | - | |
793 | | - | |
| 779 | + | |
794 | 780 | | |
795 | 781 | | |
796 | 782 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 2 | + | |
| 3 | + | |
16 | 4 | | |
17 | 5 | | |
18 | 6 | | |
| |||
0 commit comments