Skip to content

Commit 8b6a9d6

Browse files
lucianghindalucianghinda
committed
Improve the readability of bencmarks by adding the code in the Readme
1 parent 717ffa0 commit 8b6a9d6

File tree

3 files changed

+216
-41
lines changed

3 files changed

+216
-41
lines changed

‎README.md

Lines changed: 212 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,30 @@ The benchmark is focused on benchmarking the keyword arguments.
2626

2727
### Creating Values
2828

29+
Having defines the following keys and values:
30+
31+
```ruby
32+
keys = 1000.times.map { |i| "key#{i}".to_sym }
33+
values = 1000.times.map { |i| "value#{i}" }
34+
keys_and_values = Hash[keys.zip(values)]
35+
```
36+
37+
The creation benchmarks are testing the following code:
38+
39+
```ruby
40+
DataStruct = Struct.new(*keys, keyword_init: true)
41+
DataStruct.new(**keys_and_values)
42+
43+
# vs
44+
45+
DataDefine = Data.define(*keys)
46+
DataDefine.new(**keys_and_values)
47+
48+
# vs
49+
50+
OpenStruct.new(**keys_and_values)
51+
```
52+
2953
#### Benchmark with `bmbm`
3054

3155
This benchmark is run with Ruby default benchmark using `bmbm`
@@ -91,6 +115,31 @@ Comparison:
91115

92116
### Accessing Attributes
93117

118+
Having the following data defined:
119+
120+
```ruby
121+
keys = 1000.times.map { |i| "key#{i}".to_sym }
122+
values = 1000.times.map { |i| "value#{i}" }
123+
keys_and_values = Hash[keys.zip(values)]
124+
```
125+
126+
And then defining the following structures:
127+
128+
```ruby
129+
BigDataS = Struct.new(*keys, keyword_init: true)
130+
BigDataD = Data.define(*keys)
131+
```
132+
133+
The benchmarks are comparing:
134+
135+
```ruby
136+
keys.each { struct_object.send(_1) }
137+
138+
keys.each { data_object.send(_1) }
139+
140+
keys.each { opens_struct_object.send(_1) }
141+
```
142+
94143
#### Benchmark with `bmbm`
95144

96145
This benchmark is run with Ruby default benchmark using `bmbm`
@@ -131,10 +180,36 @@ Comparison:
131180
OpenStruct: 13474.6 i/s - 2.13x slower
132181
```
133182

134-
## Comparing Data.define
183+
## Data.define - benchmarks
135184

136185
### Comparing multiple ways to create a new object
137186

187+
Having the following data specified:
188+
189+
```ruby
190+
keys = 1000.times.map { |i| "key#{i}".to_sym }
191+
values = 1000.times.map { |i| "value#{i}" }
192+
keys_and_values = Hash[keys.zip(values)]
193+
194+
DataDefine = Data.define(*keys)
195+
```
196+
197+
The benchmarks are comparing the following code:
198+
199+
```ruby
200+
# Keyword arguments
201+
DataDefine.new(**keys_and_values)
202+
203+
# Positional arguments
204+
DataDefine.new(*values)
205+
206+
# Constructor method
207+
DataDefine[*values]
208+
209+
# Constructor keywords
210+
DataDefine[**keys_and_values]
211+
```
212+
138213
#### Benchmark with `bmbm`
139214

140215
This benchmark is run with Ruby default benchmark using `bmbm`
@@ -211,7 +286,39 @@ Constructor keywords: 36792 allocated - same
211286

212287
### Comparing accessing data from objects created differently
213288

214-
There is no difference in accessing attributes depending on how the object was created.
289+
Taking in consideration the following data :
290+
291+
```ruby
292+
keys = 1000.times.map { |i| "key#{i}".to_sym }
293+
values = 1000.times.map { |i| "value#{i}" }
294+
keys_and_values = Hash[keys.zip(values)]
295+
```
296+
297+
And creating the following objects:
298+
299+
```ruby
300+
DataDefine = Data.define(*keys)
301+
keyword_args = DataDefine.new(**keys_and_values)
302+
positional_args = DataDefine.new(*values)
303+
constructor_method = DataDefine[*values]
304+
constructor_with_keyword_args = DataDefine[**keys_and_values]
305+
```
306+
307+
The benchmarks are comparing the following code:
308+
309+
```ruby
310+
#Keyword arguments
311+
keys.each { keyword_args.send(_1) }
312+
313+
#Positional arguments
314+
keys.each { positional_args.send(_1) }
315+
316+
#Constructor method
317+
keys.each { constructor_method.send(_1) }
318+
319+
#Constructor keywords
320+
keys.each { constructor_with_keyword_args.send(_1) }
321+
```
215322

216323
#### Benchmark with `bmbm`
217324

@@ -262,32 +369,100 @@ Constructor keywords: 26794.4 i/s - same-ish: difference falls within error
262369

263370
In these tests I run creating new objects with 6 attributes.
264371

265-
I compared Data.define with Struct, OpenStruct, custom class with positional arguments and custom class with keyword arguments.
372+
I compared `Data.define` with `Struct`, `OpenStruct`, plain Ruby object with positional arguments and plain Ruby object with keyword arguments.
373+
374+
The tests are done only with `ips` and `bmbm`.
375+
Did not do a memory test becuase I did not wanted to try to replicate in the custom class the same logic that `Data.define` or `Struct` can offer.
266376

267-
The tests are done only with `ips` and `bmbm`. Did not do a memory test becuase I did not wanted to try to replicate in the custom class the same logic that Data.define or Struct can offer.
377+
Having defined the following data:
378+
379+
```ruby
380+
keys = [:key1, :key2, :key3, :key4, :key5, :key6]
381+
values = 6.times.map { |i| "value#{i}" }
382+
keys_and_values = Hash[keys.zip(values)]
383+
```
384+
385+
And the following classes:
386+
387+
```ruby
388+
DataStructKeyword = Struct.new(*keys, keyword_init: true)
389+
DataStructPositional = Struct.new(*keys)
390+
DataDefine = Data.define(*keys)
391+
392+
class MyValueObjectWithKeywordArgs
393+
attr_reader :key1, :key2, :key3, :key4, :key5, :key6
394+
395+
def initialize(key1:, key2:, key3:, key4:, key5:, key6:)
396+
@key1 = key1
397+
@key2 = key2
398+
@key3 = key3
399+
@key4 = key4
400+
@key5 = key5
401+
@key6 = key6
402+
end
403+
end
404+
405+
class MyValueObjectWithPositionalArgs
406+
attr_reader :key1, :key2, :key3, :key4, :key5, :key6
407+
408+
def initialize(key1, key2, key3, key4, key5, key6)
409+
@key1 = key1
410+
@key2 = key2
411+
@key3 = key3
412+
@key4 = key4
413+
@key5 = key5
414+
@key6 = key6
415+
end
416+
end
417+
```
418+
419+
The benchmarks will compare the following code:
420+
421+
```ruby
422+
# Struct - positional
423+
DataStructPositional.new(*values)
424+
425+
# Struct - keywords
426+
DataStructKeyword.new(**keys_and_values)
427+
428+
# Data - positional
429+
DataDefine.new(*values)
430+
431+
# Data - keywords
432+
DataDefine.new(**keys_and_values)
433+
434+
# OpenStruct.new
435+
OpenStruct.new(**keys_and_values)
436+
437+
# PORO - positional
438+
MyValueObjectWithPositionalArgs.new(*values)
439+
440+
# PORO - keywords
441+
MyValueObjectWithKeywordArgs.new(**keys_and_values)
442+
```
268443

269444
### Benchamrk with `bmbm`
270445

271446
```bash
272447
Creating a new object - Benchmark with bmbm - small numbers
273448
Rehearsal -------------------------------------------------------
274-
Struct - positional 0.000003 0.000000 0.000003 ( 0.000002)
449+
Struct - positional 0.000003 0.000001 0.000004 ( 0.000002)
275450
Struct - keywords 0.000002 0.000000 0.000002 ( 0.000002)
276-
Data - positional 0.000002 0.000000 0.000002 ( 0.000002)
451+
Data - positional 0.000002 0.000001 0.000003 ( 0.000004)
277452
Data - keywords 0.000002 0.000000 0.000002 ( 0.000001)
278-
OpenStruct.new 0.000020 0.000000 0.000020 ( 0.000020)
279-
Custom - positional 0.000002 0.000001 0.000003 ( 0.000002)
280-
Custom - keywords 0.000002 0.000000 0.000002 ( 0.000001)
281-
---------------------------------------------- total: 0.000034sec
453+
OpenStruct.new 0.000019 0.000001 0.000020 ( 0.000019)
454+
PORO - positional 0.000002 0.000000 0.000002 ( 0.000002)
455+
PORO - keywords 0.000001 0.000001 0.000002 ( 0.000002)
456+
---------------------------------------------- total: 0.000035sec
282457

283458
user system total real
284-
Struct - positional 0.000001 0.000001 0.000002 ( 0.000001)
285-
Struct - keywords 0.000001 0.000001 0.000002 ( 0.000001)
286-
Data - positional 0.000001 0.000000 0.000001 ( 0.000002)
459+
Struct - positional 0.000001 0.000000 0.000001 ( 0.000000)
460+
Struct - keywords 0.000002 0.000000 0.000002 ( 0.000001)
461+
Data - positional 0.000002 0.000000 0.000002 ( 0.000002)
287462
Data - keywords 0.000001 0.000000 0.000001 ( 0.000001)
288-
OpenStruct.new 0.000015 0.000000 0.000015 ( 0.000015)
289-
Custom - positional 0.000001 0.000000 0.000001 ( 0.000001)
290-
Custom - keywords 0.000001 0.000000 0.000001 ( 0.000001)
463+
OpenStruct.new 0.000018 0.000000 0.000018 ( 0.000017)
464+
PORO - positional 0.000001 0.000000 0.000001 ( 0.000001)
465+
PORO - keywords 0.000002 0.000000 0.000002 ( 0.000001)
291466
```
292467

293468
### Benchmark with `ips`
@@ -296,30 +471,30 @@ Custom - keywords 0.000001 0.000000 0.000001 ( 0.000001)
296471
Creating a new object - Benchmark with ips - small numbers
297472
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23]
298473
Warming up --------------------------------------
299-
Struct - positional 905.029k i/100ms
300-
Struct - keywords 463.944k i/100ms
301-
Data - positional 335.454k i/100ms
302-
Data - keywords 486.978k i/100ms
303-
OpenStruct.new 10.706k i/100ms
304-
Custom - positional 802.961k i/100ms
305-
Custom - keywords 475.091k i/100ms
474+
Struct - positional 913.180k i/100ms
475+
Struct - keywords 464.416k i/100ms
476+
Data - positional 335.700k i/100ms
477+
Data - keywords 484.506k i/100ms
478+
OpenStruct.new 10.869k i/100ms
479+
PORO - positional 796.702k i/100ms
480+
PORO - keywords 467.403k i/100ms
306481
Calculating -------------------------------------
307-
Struct - positional 8.766M (± 0.1%) i/s - 44.346M in 5.059002s
308-
Struct - keywords 4.565M (± 0.3%) i/s - 23.197M in 5.081195s
309-
Data - positional 3.332M (± 0.2%) i/s - 16.773M in 5.033458s
310-
Data - keywords 4.818M (± 0.2%) i/s - 24.349M in 5.053850s
311-
OpenStruct.new 107.602k (± 1.6%) i/s - 546.006k in 5.075550s
312-
Custom - positional 7.757M (± 0.3%) i/s - 39.345M in 5.072337s
313-
Custom - keywords 4.713M (± 0.5%) i/s - 23.755M in 5.040592s
482+
Struct - positional 8.924M (± 0.1%) i/s - 44.746M in 5.014174s
483+
Struct - keywords 4.583M (± 0.2%) i/s - 23.221M in 5.067154s
484+
Data - positional 3.289M (± 0.2%) i/s - 16.449M in 5.001151s
485+
Data - keywords 4.786M (± 0.1%) i/s - 24.225M in 5.061479s
486+
OpenStruct.new 109.959k (± 1.1%) i/s - 554.319k in 5.041756s
487+
PORO - positional 7.791M (± 0.3%) i/s - 39.038M in 5.010537s
488+
PORO - keywords 4.659M (± 0.5%) i/s - 23.370M in 5.016246s
314489

315490
Comparison:
316-
Struct - positional: 8765862.3 i/s
317-
Custom - positional: 7756859.0 i/s - 1.13x slower
318-
Data - keywords: 4817919.9 i/s - 1.82x slower
319-
Custom - keywords: 4712787.7 i/s - 1.86x slower
320-
Struct - keywords: 4565350.1 i/s - 1.92x slower
321-
Data - positional: 3332250.2 i/s - 2.63x slower
322-
OpenStruct.new: 107602.1 i/s - 81.47x slower
491+
Struct - positional: 8923882.2 i/s
492+
PORO - positional: 7791346.2 i/s - 1.15x slower
493+
Data - keywords: 4786215.7 i/s - 1.86x slower
494+
PORO - keywords: 4659024.3 i/s - 1.92x slower
495+
Struct - keywords: 4582626.3 i/s - 1.95x slower
496+
Data - positional: 3289111.1 i/s - 2.71x slower
497+
OpenStruct.new: 109959.0 i/s - 81.16x slower
323498
```
324499

325500

‎small_numbers_comparison/creating_values/bmbm.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,11 +59,11 @@ def initialize(key1, key2, key3, key4, key5, key6)
5959
OpenStruct.new(**keys_and_values)
6060
end
6161

62-
x.report("Custom - positional") do
62+
x.report("PORO - positional") do
6363
MyValueObjectWithPositionalArgs.new(*values)
6464
end
6565

66-
x.report("Custom - keywords") do
66+
x.report("PORO - keywords") do
6767
MyValueObjectWithKeywordArgs.new(**keys_and_values)
6868
end
6969
end

‎small_numbers_comparison/creating_values/ips.rb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,11 +59,11 @@ def initialize(key1, key2, key3, key4, key5, key6)
5959
OpenStruct.new(**keys_and_values)
6060
end
6161

62-
x.report("Custom - positional") do
62+
x.report("PORO - positional") do
6363
MyValueObjectWithPositionalArgs.new(*values)
6464
end
6565

66-
x.report("Custom - keywords") do
66+
x.report("PORO - keywords") do
6767
MyValueObjectWithKeywordArgs.new(**keys_and_values)
6868
end
6969

0 commit comments

Comments
 (0)