Skip to content

bench with fully inlined/specialized implementation #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

jmcardon
Copy link
Member

@jmcardon jmcardon commented Jan 1, 2024

This PRs to my own branch.

In my very very small benchmark of (+ 1 2) and (const 1 2), SpecializedCEK performs more than 10x better. This is definitely just GHC specialization and inlining compared to SpecializedCEK.

At the moment of the start of this PR, these are the current numbers on my M2 mac:

benchmarking pact-core-term-gas/(+ 1 2)
time                 2.270 μs   (2.235 μs .. 2.307 μs)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 2.255 μs   (2.241 μs .. 2.283 μs)
std dev              61.79 ns   (39.51 ns .. 92.55 ns)
variance introduced by outliers: 35% (moderately inflated)

benchmarking pact-core-term-gas/special (+ 1 2)
time                 177.7 ns   (177.5 ns .. 177.9 ns)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 177.9 ns   (177.7 ns .. 178.2 ns)
std dev              874.8 ps   (617.5 ps .. 1.381 ns)

benchmarking pact-core-term-gas/const unspecialized
time                 2.485 μs   (2.482 μs .. 2.488 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 2.484 μs   (2.481 μs .. 2.488 μs)
std dev              12.24 ns   (8.422 ns .. 18.02 ns)

benchmarking pact-core-term-gas/const specialized
time                 185.3 ns   (184.4 ns .. 186.1 ns)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 185.0 ns   (184.7 ns .. 185.4 ns)
std dev              1.034 ns   (702.8 ps .. 1.550 ns)

A few things SpecializedCEK does vs CEK:

  • All callsites are concretized to Eval
  • rawBuiltinEnv is used directly within the call of eval

Part of the problem with our natives which GHC may be unhappy with, is evalCEK, returnCEK and returnCEKValue being exported may not make GHC happy.

The point of this PR is make CEK.hs fast within a reasonable margin of SpecializedCEK.

@0xd34df00d
Copy link
Contributor

As the lowest-hanging fruit, just curious — how do SPECIALIZE pragmas perform on the polymoprhic CEK?

@jmcardon
Copy link
Member Author

jmcardon commented Jan 1, 2024

LOL red herring.

So it does seem there's essentially 0 difference between the two implementations when you simply swap out benchmarkEnv with benchmarkBigStepEnv.. In other words, what it seems the benchmark revealed, is that the bigstep interpreter is over 10x faster than the smallstep interpreter... lol.

So.... Embarassingly, I will close this PR. Nothing changed.. A small typo in the benchmark threw me for a loop.

@jmcardon jmcardon closed this Jan 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants