Skip to content

Commit

Permalink
Fix buffer overflow in TransformationListListNC
Browse files Browse the repository at this point in the history
To save memory, when calculating the size of the transformation,
indices where src[i]=ran[i] are skipped. Therefore these same
indices must be skipped skipped when filling in the transformation,
as we have not allocated memory for them.
  • Loading branch information
ChrisJefferson committed May 21, 2019
1 parent 4f945f0 commit ecb201b
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 4 deletions.
15 changes: 11 additions & 4 deletions src/trans.cc
Original file line number Diff line number Diff line change
Expand Up @@ -476,8 +476,12 @@ static Obj FuncTransformationListListNC(Obj self, Obj src, Obj ran)
ptf2[i] = i;
}
for (i = LEN_LIST(src); 1 <= i; i--) {
ptf2[INT_INTOBJ(ELM_LIST(src, i)) - 1] =
INT_INTOBJ(ELM_LIST(ran, i)) - 1;
s = INT_INTOBJ(ELM_LIST(src, i));
r = INT_INTOBJ(ELM_LIST(ran, i));
// deg may be smaller than s if s = r
if (s != r) {
ptf2[s - 1] = r - 1;
}
}
}
else {
Expand All @@ -487,8 +491,11 @@ static Obj FuncTransformationListListNC(Obj self, Obj src, Obj ran)
ptf4[i] = i;
}
for (i = LEN_LIST(src); 1 <= i; i--) {
ptf4[INT_INTOBJ(ELM_LIST(src, i)) - 1] =
INT_INTOBJ(ELM_LIST(ran, i)) - 1;
s = INT_INTOBJ(ELM_LIST(src, i));
r = INT_INTOBJ(ELM_LIST(ran, i));
if (s != r) {
ptf4[s - 1] = r - 1;
}
}
}
return f;
Expand Down
6 changes: 6 additions & 0 deletions tst/testinstall/trans.tst
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,12 @@ Error, TransformationListListNC: <ran> must be a small list (not a permutation\
(small))
gap> TransformationListListNC([], []);
IdentityTransformation
gap> TransformationListListNC([1..100000], [1..100000]);
IdentityTransformation
gap> TransformationListList([1..1000000], Concatenation([100000], [2..1000000]));
<transformation on 100000 pts with rank 99999>
gap> TransformationListList([1..1000000], Concatenation([2], [2..1000000]));
Transformation( [ 2, 2 ] )

# Test DegreeOfTransformation
gap> f := TransformationListListNC([1, 2], [1, 1]) ^ (3, 4);;
Expand Down

0 comments on commit ecb201b

Please sign in to comment.