Speed up linear test by using the undocumented LinearQ function.

[vsht]

Using the Internal`LinearQ routine mentioned here one can roughly gain an order of magnitude speed up when
calling LinearTest

SetDirectory[NotebookDirectory[]];
eqs = Get["Eqs.m"];

LinearTestNew[expr_] := 
  Internal`LinearQ[expr, Variables[expr /. Equal -> List]];
LinearTestOld[expr_] := 
  Return[If[
    Sort[Function[monomial, 
         Total[Map[Function[term, term[[2]]], 
           Select[FactorList[monomial], ! NumberQ[#1[[1]]] &]]]] /@ 
        MonomialList[expr]][[-1]] > 1, False, True]];

AbsoluteTiming[LinearTestNew /@ eqs]
(*{0.0172, {True, True, True, True, True, True, True, True,*)
AbsoluteTiming[LinearTestOld /@ eqs]
(*{0.111159, {True, True, True, True, True, True, True,*)

Eqs.zip

[christophmeyer]

Hi Vlad, thanks for the suggestion. Have you measured the total speedup of this change when running the examples?

[vsht]

This should mostly be relevant for rather large systems, where the number of equations is so
high that LinearTest starts to become a bottleneck.

In our test suite (I didn't run the longer examples) "Triple box" seems to benefit most from the new fix.

On Mma 13.0 (Linux) with Ryzen 4750U when running

Tests/examples.sh math "skip={K4Integral,SingleTopT1,SingleTopT2,VVT2,VVT1,DrellYanOneMass};" 

I get

* Triple box. Time estimate: 2 min
Transforming sector 1.
Transforming sector 2.
Transforming sector 3.
Transforming sector 4.
Transforming sector 5.
Transforming sector 6.
Transforming sector 7.
Transforming sector 8.
Transforming sector 9.
Transforming sector 10.
Transforming sector 11.
Transforming sector 12.
Transforming sector 13.
Transforming sector 14.
Transforming sector 15.
Transforming sector 16.
Transforming sector 17.
Transforming sector 18.
Transforming sector 19.
Check the obtained epsilon form:  CORRECT 
	CPU Time used: 219.273 s.
	RAM used: 81 MB

for the current implementation and

* Triple box. Time estimate: 2 min
Transforming sector 1.
Transforming sector 2.
Transforming sector 3.
Transforming sector 4.
Transforming sector 5.
Transforming sector 6.
Transforming sector 7.
Transforming sector 8.
Transforming sector 9.
Transforming sector 10.
Transforming sector 11.
Transforming sector 12.
Transforming sector 13.
Transforming sector 14.
Transforming sector 15.
Transforming sector 16.
Transforming sector 17.
Transforming sector 18.
Transforming sector 19.
Check the obtained epsilon form:  CORRECT 
	CPU Time used: 213.497 s.
	RAM used: 81 MB

with LinearQ. The timings also fluctuate a but, but LinearQ always seems to be faster, as I would expect.

Here is another run

log_new.txt
log_old.txt

[christophmeyer]

I am a bit hesitant about this suggestion tbh. My concern is that an undocumented function could disappear with any new patch version of Mathematica, which would then render CANONICA unusable. And even though any performance improvement is certainly nice to have, I think a few per cent don't matter that much. So unless you have seen an example where this makes a more sizeable difference, I would suggest we keep this in mind and watch the Mathematica changelog, so we can incorporate it once it becomes a documented function.

[vsht]

Well, Mathematica is not really known for being 100% backwards compatible and it is rather common
that packages stop working with a recently released version. Like we already had it with Mma 12 and 13.

What I mean is that permanent maintenance is anyhow a must to ensure the survival of the given package.
And I also don't expect Internal*`-functions becoming officially documented in the future.

Would it be acceptable for you to have a global switch, say something like $CANONICAUseFasterLinearTest that
would activate LinearQ when set to True but use the old code otherwise (and by default)? This way nothing will
potentially get broken (because of that).

[christophmeyer]

I agree that maintenance for new major version will be necessary, I just want to avoid that happening on patch updates as well. That being said, if you can add a global switch like you suggested, people can use it at their own risk, so I'd be ok with that.