Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up a benchmark for our trampolines. #519

Merged
merged 3 commits into from
Sep 15, 2015
Merged

Conversation

non
Copy link
Contributor

@non non commented Sep 9, 2015

This commit adds a new benchmark to the bench project.
It updates the version of JMH that we are using, and
also adds a new, hidden Eval subtype which improves
the performance of Eval.defer.

This commit adds a new benchmark to the bench project.
It updates the version of JMH that we are using, and
also adds a new, hidden Eval subtype which improves
the performance of Eval.defer.
@non non added the in progress label Sep 9, 2015
@non
Copy link
Contributor Author

non commented Sep 9, 2015

Here's the benchmark result running on my machine:

Benchmark                    Mode  Cnt      Score      Error  Units
TrampolineBench.eval        thrpt    3  23049.351 ± 3891.576  ops/s
TrampolineBench.stdlib      thrpt    3  11827.608 ± 1919.654  ops/s
TrampolineBench.trampoline  thrpt    3   5743.017 ±  197.309  ops/s

(Run with bench/jmh:run -i 3 -wi 3 -f1 -t1 cats.bench.TrampolineBench)

It seems like TailCalls in 2.10 lacks .flatMap and .map,
meaning that our benchmark won't work. If we comment it
out for now, we can eventually use cats.macros.Platform
to compile this benchmark only on 2.11.
@ceedubs
Copy link
Contributor

ceedubs commented Sep 10, 2015

Are the build failures random or related to this?

@non
Copy link
Contributor Author

non commented Sep 10, 2015

I'm looking into it. It doesn't seem like Travis flakiness.

@non
Copy link
Contributor Author

non commented Sep 10, 2015

So validate seems to reliably pass locally. It could be an issue of memory/resources available.

@ceedubs
Copy link
Contributor

ceedubs commented Sep 10, 2015

I restarted the builds and this time they worked in 2.10 but not 2.11. It does seem like a quirky sporadic/resource issue, but if we aren't seeing this outside of this branch, I'm also a little hesitant to merge this PR simply because I don't want us to start completely ignoring build statuses. @inthenow do you have any ideas?

@non
Copy link
Contributor Author

non commented Sep 10, 2015

Maybe we should set up Contained-Based builds? I wonder if that would help: http://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

EDIT: This article looks old -- maybe we are already doing this. I need to investigate further.

@codecov-io
Copy link

Current coverage is 65.16%

Merging #519 into master will increase coverage by +0.07% as of b4a9df2

@@            master    #519   diff @@
======================================
  Files          156     156       
  Stmts         2418    2420     +2
  Branches        68      68       
  Methods          0       0       
======================================
+ Hit           1574    1577     +3
  Partial          0       0       
+ Missed         844     843     -1

Review entire Coverage Diff as of b4a9df2

Powered by Codecov. Updated on successful CI builds.

@codecov-io
Copy link

Current coverage is 65.16%

Merging #519 into master will increase coverage by +0.07% as of f537fec

@@            master    #519   diff @@
======================================
  Files          156     156       
  Stmts         2418    2420     +2
  Branches        68      68       
  Methods          0       0       
======================================
+ Hit           1574    1577     +3
  Partial          0       0       
+ Missed         844     843     -1

Review entire Coverage Diff as of f537fec

Powered by Codecov. Updated on successful CI builds.

@non
Copy link
Contributor Author

non commented Sep 15, 2015

Awesome. We seem to have overcome the previous build flakiness!

@ceedubs
Copy link
Contributor

ceedubs commented Sep 15, 2015

👍

1 similar comment
@adelbertc
Copy link
Contributor

👍

adelbertc added a commit that referenced this pull request Sep 15, 2015
Set up a benchmark for our trampolines.
@adelbertc adelbertc merged commit 14b5772 into master Sep 15, 2015
@non non deleted the topic/eval-call-bench branch April 28, 2016 03:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants