Automatically close unused streams during garbage collection

[fingolfin]

It would be better if we automatically took care of closing streams whenever possible. I see two ways to achieve that:

1. via finalizers
2. via weak pointers

Using finalizers requires work in the kernel: basically, whenever a stream holding a file descriptor is garbage collected, we could close its file descriptor (to avoid closing the same fd twice, CloseStream then should also make sure to do something like stream![1] := -1). But finalizers can only be set in the kernel, and only for a whole TNUM at once... One could get around that in various ways, e.g. we could add a T_FILEDESC TNUM just for this... I could go on, but let's rather look at the second option first.

We could use a "weak pointer object": we already have InputTextFileStillOpen and OutputTextFileStillOpen which hold all open file descriptors. I propose we merge them into one (not strictly necessary, but reduces the work needed and the general overhead), and add a companion WeakPointerObj which tracks for each fd the corresponding stream.. So roughly like this (pseudo code):

FileDescriptorsStillOpen := []; # TODO: probably should be thread local?
StreamsStillOpen := [];

# helper for use in stream constructors
BindGlobal("RegisterFD", function(fd, stream)
  local i;
  i := 1;
  # search for an unused slot
  while IsBound(FileDescriptorsStillOpen[i]) and IsBound(StreamsStillOpen[i]) do
    i := i + 1;
  od;
  # if a stream is gone but its file descriptor is still there, close the file descriptor
  if IsBound(FileDescriptorsStillOpen[i]) then
    CLOSE_FILE(FileDescriptorsStillOpen[i]);
  fi;
  FileDescriptorsStillOpen[i] := fid;
  StreamsStillOpen[i] := stream;
end);

# helper to use in CloseStream methods
BindGlobal("CloseFD", function(fd)
  local i;
  i := Position(FileDescriptorsStillOpen, fd);
  Unbind(FileDescriptorsStillOpen[i]);
  Unbind(StreamsStillOpen[i]);
  CLOSE_FILE(fd)
end);

That does of course not make your patch redundant, but it would reduce the impact of people forgetting to use CloseStream

Originally posted by @fingolfin in #5146 (comment)

[ChrisJefferson]

I think (I'm happy to be proved wrong), it might be hard to get the TNUM version right, as we'd have to make sure there was never a way of closing a stream, re-opening the same fd, and then much later a GC occurring and the fd getting freed.

The weak pointer version seems like a good idea, and will also interact with things like Julia or the IO package getting their own file descriptors in their own way.

[fingolfin]

I don't see the problem with the TNUM version, as long as it is opaque: then it would be passed around instead of a small int representing the file descriptor; and the "close" function would simply replace the fd stored inside by an invalid bit pattern. That way a double free can't occur.

But it's a bunch of kernel code for one special case, so I'd rather avoid it