Crash when ApplicableMethod is called incorrectly

[ssiccha]

On current master 5dad0ef:

gap> obj := RandomMat(20, 20, GF(5));; 
gap> ConvertToMatrixRep(obj);
5
gap> ApplicableMethod(PostMakeImmutable, [obj]); #this works as expected
function( m ) ... end
gap> ApplicableMethod(PostMakeImmutable, obj); #this crashes
Segmentation fault

[fingolfin]

Simpler way to reproduce it (and running in a GAP debug build with assertions enabled):

gap> ApplicableMethod(PostMakeImmutable, [1..10]);
Assertion failed: (0 <= i && i < 8), function METHS_OPER, file ../../src/opers.h, line 180.
Abort trap: 6

So clearly a bounds check is missing somewhre

[cdwensley]

The second parameter for ApplicableMethod should be a list of parameters for the operation, so the first call is correct, and the second is only accepted for the convenience of users. When 'obj' is used instead of '[obj]' it happens that obj is a list (of 20 mutable compressed vectors of length 20 over GF(5)). The segmentation fault occurs when

methods:=MethodsOperation(oper,narg);

is called with narg=20. Please remind me: what is the maximum number of parameters for an operation?

[cdwensley]

Using 18 in place of 20 we get:

gap> obj := RandomMat(18,18, GF(5));;
gap> ConvertToMatrixRep(obj);
5
gap> ApplicableMethod(PostMakeImmutable, obj);
Error, Range: <last>-<first> (-7) must be divisible by <inc> (24) in
  LENGTH( meths ) at /Applications/gap/gapdev/lib/oper.g:1918 called from 
MethodsOperation( oper, narg ) at lib/methwhy.g:266 called from
CallFuncList( ApplicableMethodTypes, arg ) at lib/methwhy.g:438 called from
<function "ApplicableMethod">( <arguments> )
 called from read-eval loop at *stdin*:6
type 'quit;' to quit to outer loop

in the loop (oper.g line 1918) where len=24 and LENGTH(meths)=17:

    for i in [0, len .. LENGTH(meths) - len] do

[cdwensley]

Revised Pull Request #2948 to add a test in ApplicableMethod to check that the number of entries in the second parameter is not greater than 8. Also added a test in tst/testinstall/methwhy.tst.

[fingolfin]

I fixed the underlying problem in PR #3151. It should not be necessary to add another check to ApplicableMethod, but of course it won't hurt either.