Unit tests are located in class definition files and do not use MATLAB's testing system

[raacampbell]

This comment is associated with a openjournals/joss-reviews#7275.

General issue

Upon first glance, unit tests are located as methods within the class definition files, which is not standard practice. However, upon more careful inspection, at least some of these "unit tests" appears to be a demo function rather than an actual unit test. It would be helpful to reframe what these methods actually are and place any true unit tests into a proper unit testing framework in a single place.

Running unit tests for software development

The unit tests appear to be runnable in batch form via the script run_tests.m, but this incomplete. e.g. I spotted it misses https://github.com/LidkeLab/matlab-instrument-control/blob/main/%40MIC_GalvoAnalog/MIC_GalvoAnalog.m The tests I saw do not use MATLAB's unit testing framework, which is OOP based and contains methods for verifying outcomes using assertions and reporting results.

Are these methods really unit tests?

I realise that maybe the tests/demos are in the class files because they require hardware to be present and connected in order to run. Nonetheless, the point of unit testing is to run tests in an automated way before branch merges, major releases, or major commits.

Let's take an example: I look in the file Example_Camera.m and I see:

        function Success = unitTest()
            disp('Starting unit test...');
            obj = Example_Camera();
            obj.initialize();
            obj.setup_acquisition();
            obj.start_focus();
            pause(1);
            Success = true; % Assume success for simplicity <----- ***
            obj.abort();
            disp('Unit test completed successfully.');

This method seems to test the functioning of the camera but it has to be run interactively. There no automated test taking place here: it returns a hard-coded true for success and clearly requires user input to run. It's certainly not a bad idea to include such demo functions for users to check code is working, but calling it a unit test seems misleading since it must be run interactively. I would rename all these methods to demo or similar.

Suggestion course of action

• Create a /tests folder that outside of the code folder (see Code organisation #124). Tests can be organised hierarchically in sub-folders.
• Move any real test methods (if any) out of their class definition files and into tests.
• Rename what are really demo methods but called "unit tests" to something more meaningful.
• Organise any unit testing using MATLAB's nice testing ecosystem.