Add benchmarks for Substring.dropFirst(_:) and dropLast(_:) #32473

natecook1000 · 2020-06-19T19:12:12Z

These are self-slicing operations, so they should be fast even for this non-random-access collection.

Hoping to see a difference in this new benchmark from the change in #32451.

These are self-slicing operations, so they should be fast even for this non-random-access collection.

natecook1000 · 2020-06-19T19:12:23Z

@swift-ci Please benchmark

swift-ci · 2020-06-19T19:57:22Z

Performance: -O

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	20	20	20	—
SubstringDropLast1	15	16	15	—

Code size: -O

Regression	OLD	NEW	DELTA	RATIO
Substring.o	15635	17043	+9.0%	0.92x

Performance: -Osize

Regression	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubToNSDate2	550	630	+14.5%	0.87x (?)

Improvement	OLD	NEW	DELTA	RATIO
FlattenListLoop	2466	1632	-33.8%	1.51x (?)
FlattenListFlatMap	4908	4295	-12.5%	1.14x (?)
SubstringComparable	17	15	-11.8%	1.13x

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	21	21	21	—
SubstringDropLast1	16	16	16	—

Code size: -Osize

Regression	OLD	NEW	DELTA	RATIO
Substring.o	15754	17333	+10.0%	0.91x

Performance: -Onone

Regression	OLD	NEW	DELTA	RATIO
StringFromLongWholeSubstring	11	12	+9.1%	0.92x (?)

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	106	106	106	—
SubstringDropLast1	101	102	101	—

Code size: -swiftlibs

✅	Benchmark Check Report
⚠️⏱	`SubstringDropLast1` execution took 15 μs. _{Increase the workload of SubstringDropLast1 to be more than 20 μs.}

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

natecook1000 · 2020-06-19T21:06:09Z

@swift-ci Please smoke test and merge

xwu · 2020-06-20T00:56:09Z

benchmark/single-source/Substring.swift

+@inline(never)
+public func run_SubstringDropFirst1(_ N: Int) {
+  let s = quiteLong[...]
+  for _ in 1...N*500 {


Maybe 5000 to get a more reliable benchmark?

natecook1000 · 2020-06-22T06:12:35Z

@swift-ci Please smoke benchmark

natecook1000 · 2020-06-22T06:12:48Z

@swift-ci Please smoke test

swift-ci · 2020-06-22T07:32:15Z

Performance: -O

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	32	32	32	—
SubstringDropLast1	25	26	26	—

Code size: -O

Regression	OLD	NEW	DELTA	RATIO
Substring.o	15635	17155	+9.7%	0.91x

Performance: -Osize

Improvement	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubToNSDateRef	2360	2200	-6.8%	1.07x (?)

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	29	29	29	—
SubstringDropLast1	24	26	25	—

Code size: -Osize

Regression	OLD	NEW	DELTA	RATIO
Substring.o	15754	17315	+9.9%	0.91x

Performance: -Onone

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	144	144	144	—
SubstringDropLast1	140	145	142	—

Code size: -swiftlibs

✅	Benchmark Check Report
⛔️⏱	`SubstringDropFirst1` has setup overhead of 2 μs (6.3%). _{Move initialization of benchmark data to the setUpFunction registered in BenchmarkInfo.}

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: 6-Core Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB

natecook1000 · 2020-06-24T18:38:49Z

@swift-ci Please smoke benchmark

natecook1000 · 2020-06-24T19:42:34Z

@swift-ci Please smoke benchmark

swift-ci · 2020-06-24T20:14:33Z

Performance: -O

Regression	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubToNSDate2	540	620	+14.8%	0.87x (?)
ObjectiveCBridgeStubToNSDateRef	3500	3940	+12.6%	0.89x (?)
ObjectiveCBridgeFromNSArrayAnyObjectForced	4340	4860	+12.0%	0.89x (?)

Improvement	OLD	NEW	DELTA	RATIO
NSStringConversion.MutableCopy.Rebridge	1020	931	-8.7%	1.10x (?)
NSStringConversion.MutableCopy.Rebridge.Medium	646	599	-7.3%	1.08x (?)

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	42	42	42	—
SubstringDropLast1	32	32	32	—

Code size: -O

Regression	OLD	NEW	DELTA	RATIO
Substring.o	15635	17155	+9.7%	0.91x

Performance: -Osize

Regression	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubToNSDate2	550	620	+12.7%	0.89x (?)

Improvement	OLD	NEW	DELTA	RATIO
NSStringConversion.MutableCopy.UTF8	924	846	-8.4%	1.09x (?)

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	42	42	42	—
SubstringDropLast1	32	32	32	—

Code size: -Osize

Regression	OLD	NEW	DELTA	RATIO
Substring.o	15754	17315	+9.9%	0.91x

Performance: -Onone

Regression	OLD	NEW	DELTA	RATIO
ObjectiveCBridgeStubToNSDate2	630	710	+12.7%	0.89x (?)

Added	MIN	MAX	MEAN	MAX_RSS
SubstringDropFirst1	215	215	215	—
SubstringDropLast1	203	212	207	—

Code size: -swiftlibs

✅	Benchmark Check Report

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

natecook1000 · 2020-06-24T21:16:41Z

@swift-ci Please smoke test

Add benchmarks for Substring.dropFirst(_:) and dropLast(_:)

068ef83

These are self-slicing operations, so they should be fast even for this non-random-access collection.

xwu reviewed Jun 20, 2020

View reviewed changes

Fiddle with the iteration count and length

3aeead4

natecook1000 force-pushed the substring_dropfirst branch from c2144ae to 3aeead4 Compare June 24, 2020 18:38

natecook1000 merged commit 9855496 into swiftlang:master Jun 25, 2020

natecook1000 deleted the substring_dropfirst branch June 25, 2020 15:53

natecook1000 mentioned this pull request Jun 25, 2020

Switch to benchmark remove* instead of drop* #32551

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add benchmarks for Substring.dropFirst(_:) and dropLast(_:) #32473

Add benchmarks for Substring.dropFirst(_:) and dropLast(_:) #32473

Uh oh!

natecook1000 commented Jun 19, 2020

Uh oh!

natecook1000 commented Jun 19, 2020

Uh oh!

swift-ci commented Jun 19, 2020

Uh oh!

natecook1000 commented Jun 19, 2020

Uh oh!

xwu Jun 20, 2020

Uh oh!

natecook1000 commented Jun 22, 2020

Uh oh!

natecook1000 commented Jun 22, 2020

Uh oh!

swift-ci commented Jun 22, 2020

Uh oh!

natecook1000 commented Jun 24, 2020

Uh oh!

natecook1000 commented Jun 24, 2020

Uh oh!

swift-ci commented Jun 24, 2020

Uh oh!

natecook1000 commented Jun 24, 2020

Uh oh!

Uh oh!

Add benchmarks for Substring.dropFirst(_:) and dropLast(_:) #32473

Add benchmarks for Substring.dropFirst(_:) and dropLast(_:) #32473

Uh oh!

Conversation

natecook1000 commented Jun 19, 2020

Uh oh!

natecook1000 commented Jun 19, 2020

Uh oh!

swift-ci commented Jun 19, 2020

Performance: -O

Code size: -O

Performance: -Osize

Code size: -Osize

Performance: -Onone

Code size: -swiftlibs

Uh oh!

natecook1000 commented Jun 19, 2020

Uh oh!

xwu Jun 20, 2020

Choose a reason for hiding this comment

Uh oh!

natecook1000 commented Jun 22, 2020

Uh oh!

natecook1000 commented Jun 22, 2020

Uh oh!

swift-ci commented Jun 22, 2020

Performance: -O

Code size: -O

Performance: -Osize

Code size: -Osize

Performance: -Onone

Code size: -swiftlibs

Uh oh!

natecook1000 commented Jun 24, 2020

Uh oh!

natecook1000 commented Jun 24, 2020

Uh oh!

swift-ci commented Jun 24, 2020

Performance: -O

Code size: -O

Performance: -Osize

Code size: -Osize

Performance: -Onone

Code size: -swiftlibs

Uh oh!

natecook1000 commented Jun 24, 2020

Uh oh!

Uh oh!