[SR-9907] DataAppendDataLargeToLarge benchmark is unstable

[eeckstein]

Previous ID	SR-9907
Radar	None
Original Reporter	@eeckstein
Type	Bug
Status	Closed
Resolution	Done

Attachment: Download

Additional Detail from JIRA

Votes	0
Component/s	Compiler
Labels	Bug
Assignee	@palimondo
Priority	Medium

md5: aafd9c21b73a0959141b6a2153ce8d03

relates to:

• SR-9911 Data.append - high performance fluctuation

Issue Description:

no-op changes report a performance difference of 1.35x.

For example, see the benchmark run in #22341

[eeckstein]

Pavol, would you be able to take a look?

[palimondo]

I’ll try… I’ve seen this jumping up and down a lot. But I suspect it is an artifact of Data’s implementation. I’ll report tomorrow.

[palimondo]

OK, so DataAppendDataLargeToLarge has multimodal runtimes (2 major, and 2 minor modes). There is 36% difference between the major modes. It has about 3:10 or 4:12 chance of not hitting the minimal mode.

This multimodal behavior of Data should be looked at separately. But I think the most robust way to fix sporadic reports of this false change is to bump unchanged_lenght_count comparison in run_smoke_bench from 5 to 10. Would you agree, @eeckstein?

[palimondo]

PR #22546