special-case linear algebra operations for small matrices

[JeffBezanson]

We now know inline code is faster than BLAS for 2x2 at least.
Probably also helps for matrix*vector and backslash.

[ViralBShah]

Addressed in commit cced0a3. We probably need to reorganize the matrix multiplication code to allow multiple cases and dispatching to various different versions.

[ViralBShah]

I feel that the way to do this would be through a simple codegen, like in some of the N-d stuff, which generates the correct loops.

[StefanKarpinski]

Should we perhaps have small matrix classes so that we can more easily write specialized code for them?

[ViralBShah]

A small matrix class could be useful. But, first, we need to come up with a list of functions that we want to work fast on small matrices. Then, we need to see if it is possible to beat the library functions, and have a general method that works for small matrices, and not just 2x2 and 3x3.

In any case, you still have to put the size check in the routines, unless the small matrices are created with special constructors.

[JeffBezanson]

No, I'm pretty sure a small matrix type would be annoying. Size checks are extremely cheap, and the only reasonable way to implement this.

[StefanKarpinski]

I agree with present Jeff and disagree with past Stefan.

[ViralBShah]

Closing this one since there isn't anything really to do here now. We can reopen if somebody needs some specific small matrix operations.