kernel: refactor how (im)mutability is tracked in the kernel

[fingolfin]

This issue is about tracking a project @markuspf, myself and others have been looking at on and off in the past 1-2 years regarding the kernel: Right now, mutability of internal lists and records is stored as part of the TNUM. We would like to change that, and instead store this via an "object flag" (see SET_OBJ_FLAG in src/objects.h).

One motivation for that is that we'd like to enforce the following invariant in the kernel: "an immutable object can never become mutable". This was violated in several cases in the past, and I'd not be surprised to learn that it still is (though I think we fixed all instances we found). As long as (im)mutability is stored as part of the TNUM, loosing immutability is an easy bug to introduce, namely whenever you retype a bag, which some code does frequently. By putting the (im)mutability state into an object flag in the bag header, this cannot happen; to change mutability of an internal object, you'd have to call SET_OBJ_FLAG (or a suitable wrapper for it) explicitly, making it stand out.

Another (minor) motivation is that this reduces the number of used TNUMs considerably, making room for more future extensions (but this is hardly pressing at this point ;-).

The very rough plan to get there consists of these parts:

1. reduce use of the IMMUTABLE constant as much as possible, by replacing it with higher level functions where possible.
2. introduce a new object flag tracking (im)mutability, but keep the TNUMs as they are (one decision to make there is to whether the flag, if set, indicates mutability, or immutability; there are pros and cons either way, but really, as long as it is hidden behind a suitable set of accessor functions, it doesn't matter much)
3. add assertions in strategic points which checks that the TNUM and the mutability flag always agree
4. track down any violations revealed by this and fix them
5. finally get rid of IMMUTABLE and the immutable tnums (and various related things, like the NEXT_ENUM_EVEN macro)

Some specific TODOs (in addition to those obvious from the "plan" above):

• for HPC-GAP, once the TNUM does not indicate immutability anymore, MakeBagTypePublic cannot be used to automatically migrate immutable list and record bags to the public region. No big deal, though (and I actually like that we can thus remove some logic from RetypeBag)
• IsMutableObjFuncs likely can and should be removed once we get rid of the IMMUTABLE bit in the TNUM
• it might be useful to have a faster variant of MakeImmutable, which is only useful for builtin lists and record, and directly modifies the TNUM (now) resp. object flag (later)
• some code deals with creating a new object with mutability identical to another object; could have a function for that, too?

Some related PRs (some merged, some obsolete): #1520, #1560, #1563, #1570, #1571

Relates issues resp. bug fixes: #1632, #1637

[fingolfin]

One measure we used to gauge in parts the amount of work left is to count the number of occurrences of the word IMMUTABLE in the kernel sources, via git grep -w IMMUTABLE src | wc -l. This is 1417 for stable-4.8, down to 567 for stable-4.9, and down to 481 for master.

UPDATE: now 441 in stable-4.10, and 393 in master

The following script removes lines containing IMMUTABLE which will "obviously" become redundant once the switch to the object flag is complete; running it right now get us down to 237 (and reveals dozens more "obviously" redundant uses)

files:=[];
for dir in [ "src", "src/hpc" ] do
    dir:=Directory(dir);
    fs:=DirectoryContents(dir);
    fs:=Filtered(fs, f->EndsWith(f,".c"));
    Append(files, List(fs, f -> Filename(dir,f)));
od;


for f in files do
    Print("Processing ", f, "\n");
    data:=StringFile(f);
    lines:=SplitString(data,"\n");
    i := 2;
    while i <= Length(lines) do
        if PositionSublist(lines[i], "IMMUTABLE") = fail then
            i := i + 1;
            continue;
        fi;
        str := ReplacedString(lines[i], "+IMMUTABLE", "          ");
        if str = lines[i-1] then
            Remove(lines, i);
        fi;
        i := i + 1;
    od;
    out:=JoinStringsWithSeparator(lines, "\n");
    Add(out,'\n');
    FileString(f,out);
od;