[Perf] Linux/arm64: 1 Regression on 6/26/2026 9:56:31 PM +00:00

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	azurelinux 3.0
Queue	CobaltAzureLinux
Baseline	11b42525ceba2a40d5cd89ebc98e9255780c1949
Compare	62404420e54ff7257b87ca8cebb97673d4c42b84
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
GetByteCount - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	37.70 ns	40.24 ns	1.07	0.23	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Details

System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[LoopedBard3]

🔍 Automated Triage Analysis

Summary: All reported changes appear to be measurement noise. The single test (System.Text.Tests.Perf_Encoding.GetByteCount) shows +6.73% but has a bi-modal historical distribution (CV=11.33%, z=0.63) making this delta well within normal variance. Finding confidence 4/5 (strong historical data, 295 runs), regression confidence 1/5 (noise classification, delta far below adaptive threshold of 22.66%). Bisection found no change for 1 test(s) — it ran but the change didn't reproduce (noisy or bimodal pattern).

Finding Confidence: 4/5 (analysis accuracy)
Regression Confidence: 1/5 (likelihood of true regression)
⚠️ Assessment: Likely noise - recommend closing or monitoring for recurrence.

🤖 Proposed Next Actions: Likely noise — close & label — 1 done, 2 dry-run (rule R1_all_noise — see Full Analysis for the table). · 🦆 Re-analysis: ✅ concurs

📊 Full Analysis (click to expand)

Summary: All reported changes appear to be measurement noise. The single test (System.Text.Tests.Perf_Encoding.GetByteCount) shows +6.73% but has a bi-modal historical distribution (CV=11.33%, z=0.63) making this delta well within normal variance. Finding confidence 4/5 (strong historical data, 295 runs), regression confidence 1/5 (noise classification, delta far below adaptive threshold of 22.66%).

Issue Overview

• Issue: [Perf] Linux/arm64: 1 Regression on 6/26/2026 9:56:31 PM +00:00 #75481 - [Perf] Linux/arm64: 1 Regression on 6/26/2026 9:56:31 PM +00:00
• Date: 2026-06-26 | Queue: AzureLinux.3.Cobalt.Arm64.Perf | OS/Arch: linux/arm64
• Commits: 11b42525ceba → 62404420e54f (15 commits)
• Labels: untriaged, perf-regression, os-linux, branch-refs/heads/main, arch-arm64, runtime-coreclr, kind-micro, compilationmode-tiered, runkind-micro
• Historical Data: Available (295 prior runs) — sufficient data but high variance (bi-modal distribution)
• Runtime Stack: Desktop + CoreCLR (implicit)

Test Analysis

Test Name	Baseline	Compare	Δ%	μ	σ	n	z	Thresh%	Noise?	Assessment
System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")	37.70 ns	40.24 ns	+6.73%	37.57	4.26	295	0.63	22.66%	Y	Bi-modal noise; compare value within historical high cluster

Legend: μ=mean, σ=stddev, n=runs, z=z-score, Thresh%=adaptive threshold

Finding Confidence: 4/5 - Strong historical data (295 prior runs), complete commit info, clear statistical picture despite high variance. Bi-modal pattern well-documented.

Candidate Stack Relevance

Commit	Touched Paths (summary)	Relevance	Notes
3c2aaa4b669c	src/coreclr/jit/	Relevant	ARM64 compare instructions; could theoretically affect encoding codegen
368146691beb	src/coreclr/jit/	Relevant	Loop inversion change; could affect tight encoding loops
0bc7eda42ade	src/coreclr/	Relevant	Generic lookups/thunks fix
562ff6630e21	PAL layer cleanup	Shared	Removes stale includes; unlikely functional impact
d24159acdf71	src/coreclr/jit/ (wasm focus)	LessLikely	WASM-focused JIT fixes
62404420e54f	src/coreclr/jit/ (wasm focus)	LessLikely	WASM-specific LDVIRTFTN change

Noise Classification

System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8") (+6.73%)

• Delta: 37.70 ns → 40.24 ns (+6.73%)
• Historical Context: μ=37.57 ns (σ=4.26 ns, n=295); z-score=0.63; adaptive threshold=22.66%
• Assessment: Noise — The test exhibits a persistent bi-modal distribution (~23-25 ns low cluster and ~37-40 ns high cluster). The CV of 11.33% reflects this pattern. The compare value of 40.24 ns falls squarely within the historical high cluster. The z-score of 0.63 is far below the 2.0 significance threshold, and the delta of 6.73% is well below the adaptive threshold of 22.66%.
• Classification Rationale: All 36 post-regression runs also fall in the high cluster (~39-40 ns), consistent with the historical high band rather than indicating a new regression.
• Finding Confidence: 4/5 - Excellent historical data (295 runs), clear bi-modal pattern documented
• Regression Confidence: 1/5 - Delta well within normal variance, z-score insignificant, bi-modal noise pattern

Open Questions

• Historical Data: High variance (σ/μ=11.33%) due to persistent bi-modal distribution on ARM64 Cobalt hardware
• Confidence Limitations: None — data is sufficient to confidently classify as noise
• Verification Needed: None warranted for noise classification
• Bisection Status: Not performed (noise classification)

Recommended Actions

1. No investigation needed - The delta is measurement noise within the test's known bi-modal behavior pattern
2. Consider test reliability - This benchmark shows persistent bi-modal behavior (~23 ns vs ~40 ns clusters) on ARM64 Cobalt, possibly due to environment or CPU scheduling interference. Pinning CPU affinity or increasing iteration count may reduce variance.
3. Close as noise - This issue can be closed as the reported change is not a genuine regression

Bisection Results

Test	Status	Culprit Commit	Relevance	Reported Diff (issue)	Bisected Diff (at culprit)	Duration
GetByteCount(size: 512, encName: "utf...	🔍 No change found	-	-	37.7→40.2 ns (+7%)	— (not measured)	5014s

Total bisection time: 5014s
Success rate: 0/1 tests

System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")

• Status: inconclusive

Helix jobs

Job ID	Submitted by bisection	Last status
02031622-dfe7-47dc-97a6-a673d0a9a581	✅	completed
745edbfc-147b-4cc1-bc59-b0cf4918a865	✅	completed
fec727f9-fc6a-4d2a-b367-92530ea3d08a	✅	completed
336a34d2-b6e2-4c7d-b7b2-a2e47e171075	✅	completed
ea80c6c1-a2a9-411e-8582-b58eb7ac6b4c	✅	completed
de0dc196-c285-4838-99f0-20b88dc3d3a3	✅	completed
441f9db1-a979-403b-a90c-fc2267c698ac	✅	completed
e284c2c3-2be0-4ded-a06f-abff3c33819c	✅	completed

Bisection details

…[earlier output omitted]

● run_with_build_at_hash (MCP: PerfTriageMcp) · buildAtHash: [1 items], testName: [1 items]
└ No report found.

I've exhausted all reasonable approaches. Every Helix job across multiple configurations (arm64/x64, net9.0/net10.0/net11.0, different queues, different test name formats, different tests entirely) completes but produces no report. This is a systemic infrastructure issue — the build artifacts for these commits appear unavailable to the Helix harness.

BISECTION FAILED:
- Reason: Repeated empty results — all Helix jobs complete with "no report found" across all configurations
- Helix Errors: Every job returns {"status":"completed","message":"Job completed but no report found.","report":null}
- What was attempted:
  - Job 02031622: arm64/net11.0/AzureLinux.3.Cobalt.Arm64.Perf, 3 commits (double bisect) → no report
  - Job 745edbfc: arm64/net11.0, 2 commits → no report
  - Job fec727f9: arm64/net11.0, 2 commits → no report
  - Job 336a34d2: arm64/net10.0, baseline+compare commits → no report
  - Job ea80c6c1: arm64/net10.0, class-only test name → no report
  - Job de0dc196: x64/net10.0/default queue → no report
  - Job 441f9db1: arm64/net10.0, wildcard test name → no report
  - Job e284c2c3: arm64/net9.0, baseline+compare commits → no report
  - RunWithBuildAtHash (blocking): x64/default, multiple test names → no report
  - Tried completely different test (Perf_Dictionary) → no report
- What is needed: Investigate why build artifacts for commits 11b42525ceba..62404420e54f are not available to the Helix harness, or verify that the build-at-hash mechanism is functional for this commit range

HELIX JOBS:
- 02031622-dfe7-47dc-97a6-a673d0a9a581
- 745edbfc-147b-4cc1-bc59-b0cf4918a865
- fec727f9-fc6a-4d2a-b367-92530ea3d08a
- 336a34d2-b6e2-4c7d-b7b2-a2e47e171075
- ea80c6c1-a2a9-411e-8582-b58eb7ac6b4c
- de0dc196-c285-4838-99f0-20b88dc3d3a3
- 441f9db1-a979-403b-a90c-fc2267c698ac
- e284c2c3-2be0-4ded-a06f-abff3c33819c

🤖 Proposed Next Actions

Rule: R1_all_noise — Likely noise — close & label.

All reported regressions were classified as noise.

#	Action	Status	Details
1	Add label	✅ Done	noise
2	Remove label	⏭️ Skipped (dry-run)	untriaged
3	Close issue	⏭️ Skipped (dry-run)	reason not_planned

🦆 Pre-bisection Re-analysis (advisory)

⚠️ Disputes the pre-bisection plan — advisory only, not applied

Flags: wasted-bisection-slot

Rationale: The single finding is classified as noise with high finding confidence (4/5) and very low change confidence (1/5), yet it is still queued for bisection. Combined with bimodal history being flagged, this strongly suggests measurement artifact rather than a real regression — spending ~60 Helix minutes bisecting a 1/5 change-confidence test with known bimodal behavior is wasteful.

Suggested drop from bisection: System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8") (advisory — not applied)

This critique is advisory in the current configuration and did not change which tests were bisected.

🦆 Re-analysis (advisory)

✅ Agrees with the analysis — advisory only, not applied

Rationale: The analysis is internally consistent: bimodal history and an inconclusive bisection fully justify classifying the signal as noise with high finding confidence and low change confidence, and no culprit is claimed where none was surfaced.

This critique is advisory in the current configuration and did not change the confidences, culprit, or proposed actions above.

---

Generated by PerfTriageAgent on 2026-07-02 11:04 UTC