-
Notifications
You must be signed in to change notification settings - Fork 66
Comparison Mocking frameworks
THIS IS AN OLD DOCUMENT FROM THE INTERNET ARCHIVE
Disclaimer: These allegations need to be verified with the authors of the respective mocking frameworks. I also would like a code example for each framework worked out to allow users to compare the amount of work. Also realise that I’ve made my framework because I consider it to be a good solution. This comparison will be tainted toward my own as I designed my own mocking framework to be the way I like it to be and most others were not designed with my personal preferences in mind.
The “Has explicit states” value indicates whether there are separate Record and Playback phases. Mocking frameworks without these are slightly more flexible as you can set up some expectations, use them to create more objects and set some more expectations afterwards.
- Has explicit states: Yes
- Portability: Very good
- Default behaviour when no expectation is set: Not relevant
Mockpp replaces the original class by letting the user derive a new class, using mock-framework provided replacement helpers to make the replacing easier. The user has to replace the entire class when making a mock and a full mock class is created in its place. Recording of calls is very intuitive and constructive using the chaining syntax. It requires fairly high up-front cost to integrate and it somewhat hinders changing code, as twice as much code needs to be changed for each interface change. Mockpp requires considerably more work than using hippo mocks, at the advantage that it stays within the limits of the language. What Hippo Mocks does is slightly over the boundaries of what the language allows which means that on platforms with fully conforming compilers it might not work. Mockpp will keep working as long as your platform is fully conforming and probably quite a bit beyond that.
- Has explicit states: No
- Portability: Moderate to good
- Default behaviour when no expectation set: Throw exception Amop replaces each class on-request inside the test function. It creates a list of mocked functions per mocked object. Each mock object is a stack-based object that is cleaned up automatically. Amop uses the same techniques as Hippo Mocks and shares the same base issues. The advantage Hippo Mocks has is that expectations are always checked and that it requires no library setup for linking as it’s fully in one header. The syntax also slightly differs with my personal bias being toward Hippo Mocks.
- Has explicit states: Yes
- Portability: Fairly bad
- Default behaviour when no expectation set: Original function call Mockitnow replaces the functions by hacking the first few instructions aside and putting a jump in to its own function. Inspects function to see if it’s a virtual function thunk and handles it differently to keep polymorphism working. The syntax is awkward to use and it is very portability limited. On the plus side, it easily replaces C functions and the same basic techniques are very applicable on other systems as well. Porting this should not be too much of a hassle, but as far as I can tell it hasn’t been done.
- Has explicit states: No
- Portability: I need to take more time to look at M0cxx0r. I can’t seem to find out easily how they make it work. I think it’s in the category of Amop, Hippo Mocks and Mockitopp
- Has explicit states: ?
- Portability: Very very good
- Default behaviour when no expectation set: ? Moxy generates new classes from the source of the original. It reads this using Python and generates new output files with mock classes inside them. This sounds like a great idea, but limits your use of the language to what Moxy understands. Given that C++ is nearly provably unparseable, I strongly suspect the parser having some limitations compared to pure C++. Not to mention that C++ can still be strongly obfuscated by macros. The approach is very portable and natural as it doesn’t break in on the language, but the way of developing requires a significant change in the build procedure.
- Has explicit states: Probably does .
- Portability: Fairly well .
- Default behaviour when no expectation set: Return default-constructed. Mockitopp uses a similar technique to Amop and Hippo Mocks by replacing the contents of the class at runtime by trickily constructed compile-time code. It uses similar syntax to both and has similar applicability. Its downsides are the amount of dependencies and the fact that it has to be compiled to a library itself as well. The syntax has a few points of elegance in repeated calls that Hippo Mocks doesn’t do.
- Has explicit states: No
- Portability: Very very good
- Default behaviour when no expectation set: Throws exception Google Mock replaces your class using a set of macros that create a mock class similar to your own. This is subject to maintenance problems as your own class is as well. The technique is very portable as it introduces no compiler or platform dependencies. The expectations are stored using a very versatile expectation language that has a few specifics, which are documented very well on the Google Mock website. Compared to Hippo Mocks it has more dependencies and it requires you to make your own mocks, as well as link to its library. Not cleaning up behind yourself doesn’t show up either, as you’ll just miss not matching expectations.
- Has explicit states: No
- Portability: Very good (For hardware Nios, uBlaze, X86, ARM all have been tested and for compilers, Comeau, GCC and MSVC have been tested to find no trouble)
- Default behaviour when no expectation set: Throws exception Hippo Mocks replaces your class using an object created locally that matches the ABI specification for your interface. It replaces functions at runtime using compile-time created replacements and can show the entire information from the repository when an expectation fails. Upon leaving your test function the repository is destroyed, which checks the expectations as well and throws an exception if the test went ok up to this point. It has no external dependencies and it has no library to create or link against. The code is even in one file, so you don’t even have to change any include paths.