AMS Benchmark tests

[joaomdmoura]

AMS represents a crucial part at an application and it can impact performance (good or badly).
The goal is to make AMS an tool that not only helps you to bring convention over configuration to your JSON generation, but also give you the tools that might help your apps performance.

In order to do this would be great to have some benchmak test that help us to evaluate how much a feature / bug_fix / refactoring impact overall performance. It would be great to keep track of the difference between versions, and help to convince people to update their AMS.

I'm not so sure how would be the best way to do this, I'm thinking about how would be possible to add it to the workflow with our already existing tests. Before start something would be great to hear some thoughts about the idea, if it's valid or not, and how to implement it.

[joshsmith]

I think it's a great idea, especially after seeing this last week: https://is-ember-fast-yet.firebaseapp.com/

I don't feel one way or another about the implementation of it, but I think in general the concept is a good one. 👍

[joaomdmoura]

Great! @kurko also liked the concept. I'll put it in my queue 😄
btw #833 is exactly the kind of thing that would make a benchmark suite amazing

[joshsmith]

That's a great example @joaomdmoura.

What are your thoughts about how to implement this?

[joaomdmoura]

Right now I've 2 different implementation on my mind.

1. Use a new file to keep history and Benchmark module (link). This implementation will take into account the code it self.
2. Use and external service like Loader.io, that is freemium and simulate real requests.

I'm tending to the first option, but either way, I'm thinking about move this to a new OS project. A gem that helps you to built a benchmark test suite. Then add it as dependency to AMS.

What do you think about it @joshsmith ?

[joshsmith]

I would prefer the first option, I think, (or something like it) since that seems closer to our code and doesn't rely on any external service.

Have you also seen https://github.com/rails/rails-perftest? Not sure if this is appropriate or not.

[joaomdmoura]

Agreed. I didn't knew it, but seem great. I'll check it out to see if we can use it or at least have some ideas.