Initializing reference objects in non-default storage

[HertzDevil]

Reference#allocate, i.e. @[Primitive(:allocate)], does a lot of work behind the scenes before the allocated object's #initialize gets called:

# pseudo-code representation of `Crystal::CodeGenVisitor#allocate_aggregate`
class Reference
  def self.allocate : self
    {% if ... %} # type contains any inner pointers
      obj = __crystal_malloc64(instance_sizeof(self).to_u64!).as(self)
    {% else %}
      obj = __crystal_malloc_atomic64(instance_sizeof(self).to_u64!).as(self)
    {% end %}

    Intrinsics.memset(obj.as(Void*), 0_u8, instance_sizeof(self), false)

    {% for ivar in @type.instance_vars %}
      {% if ivar.has_default_value? %}
        pointerof(obj.@{{ ivar }}).value = {{ ivar.default_value }}
      {% end %}
    {% end %}

    set_crystal_type_id(obj)

    obj
  end
end

This implementation ties object creation to a global allocator, which is either LibC or LibGC depending on whether -Dgc_none was specified during compilation. But there is actually nothing wrong with using storage allocated by something else:

# heapapi.cr
lib LibC
  HEAP_ZERO_MEMORY = 0x00000008

  fun GetProcessHeap : HANDLE
  fun HeapAlloc(hHeap : HANDLE, dwFlags : DWORD, dwBytes : SizeT) : Void*
  fun HeapFree(hHeap : HANDLE, dwFlags : DWORD, lpMem : Void*) : BOOL
end

class Foo
  @x : Int32
  @y = "abc"

  def initialize(@x : Int32)
  end

  def self.new(x : Int32, &)
    obj = LibC.HeapAlloc(LibC.GetProcessHeap, LibC::HEAP_ZERO_MEMORY, instance_sizeof(self)).as(self)

    pointerof(obj.@y).value = "abc" # ???

    set_crystal_type_id(obj)

    begin
      obj.initialize(x)
      yield obj
    ensure
      # manual memory management!
      obj.finalize if obj.responds_to?(:finalize)
      LibC.HeapFree(LibC.GetProcessHeap, 0, obj.as(Void*))
    end
  end
end

# note the different addresses
Foo.new(1) { |foo| p foo } # => #<Foo:0x1d58eb9bc70 @x=1, @y="abc">
Foo.new(2) { |foo| p foo } # => #<Foo:0x1d58eb9c230 @x=2, @y="abc">
p Foo.new(3)               # => #<Foo:0x1d590620f60 @x=3, @y="abc">
p Foo.new(4)               # => #<Foo:0x1d590620e40 @x=4, @y="abc">

Or even on the stack, as some people have always dreamed:

class Foo
  @x : Int32
  @y = "abc"

  def initialize(@x : Int32)
  end

  def self.new(x : Int32, &)
    buf = uninitialized UInt8[instance_sizeof(self)] # alignment not guaranteed
    obj = buf.to_unsafe.as(self)

    buf.fill(0)
    pointerof(obj.@y).value = "abc" # ???
    set_crystal_type_id(obj)

    begin
      obj.initialize(x)
      yield obj
    ensure
      obj.finalize if obj.responds_to?(:finalize)
    end
  end
end

Foo.new(1) { |foo| p foo } # => #<Foo:0x100bffacc @x=1, @y="abc">
Foo.new(2) { |foo| p foo } # => #<Foo:0x100bffaa4 @x=2, @y="abc">
p Foo.new(3)               # => #<Foo:0x10bbb840f60 @x=3, @y="abc">
p Foo.new(4)               # => #<Foo:0x10bbb840e40 @x=4, @y="abc">

In these cases we have manually expanded the instance variable initializers and the set_crystal_type_id call, but all of this could have been done by a compiler primitive, because it is really just @[Primitive(:allocate)] without the allocation. So I would like to have a new primitive class method for this, say @[Primitive(:pre_initialize)]:

class Foo
  def self.new(x : Int32, &)
    obj = LibC.HeapAlloc(LibC.GetProcessHeap, 0, instance_sizeof(self)).as(self)
    Foo.pre_initialize(obj)

    begin
      obj.initialize(x)
      yield obj
    ensure
      obj.finalize if obj.responds_to?(:finalize)
      LibC.HeapFree(LibC.GetProcessHeap, 0, obj.as(Void*))
    end
  end

  def self.new(x : Int32, &)
    buf = uninitialized UInt8[instance_sizeof(self)] # alignment not guaranteed
    obj = buf.to_unsafe.as(self)
    Foo.pre_initialize(obj)

    begin
      obj.initialize(x)
      yield obj
    ensure
      obj.finalize if obj.responds_to?(:finalize)
    end
  end
end

Although the discussion is based on reference types, this would also work for value types except that set_crystal_type_id isn't called. (Non-abstract value types are leaf types and therefore do not carry the type ID in their layout. Abstract value types are taken care of via upcasting.)

I believe this will make custom allocators easier to write in Crystal if we ever go by that route, especially when targetting embedded devices.

[straight-shoota]

I suppose an alternative could be to allow passing a block to allocate for custom allocation. I don't think it has any noteworthy benefit, though.

[straight-shoota]

This is similar to #13368 and I figure the proposed .pre_initialize method could also make a good solution for that problem.

In this context I realized that it might be a good idea for .pre_initialize to receive a memory pointer and return self.
I think this makes a lot of sense because it means that this method turns a blob of memory into an object.

The usage example would be come this:

  def self.new(x : Int32, &)
    mem = LibC.HeapAlloc(LibC.GetProcessHeap, 0, instance_sizeof(self))
    obj = Foo.pre_initialize(mem)

    begin
      obj.initialize(x)
      yield obj
    ensure
      obj.finalize if obj.responds_to?(:finalize)
      LibC.HeapFree(LibC.GetProcessHeap, 0, mem)
    end
  end

I reckon we might consider a different, better fitting name.
Perhaps it could be an overload of .allocate with a parameter for the memory location?

[HertzDevil]

Indeed one of the goals of having .pre_initialize is to remove the manual calls to Object.set_crystal_type_id. Having the method return the object directly might also be useful because the object does not have to be located at the root of the allocated memory, e.g. to satisfy a stronger alignment.

[straight-shoota]

the object does not have to be located at the root of the allocated memory, e.g. to satisfy a stronger alignment.

Not sure I understand this aspect. Do you mean for .pre_initialize to advance the pointer if it's not aligned? That would be problematic with regards to the size of allocated memory.

[HertzDevil]

Well, can you think of any other instance where obj is not equal to mem.as(Foo)?

[straight-shoota]

My point is about including the cast into the method so it happens in the same space as setting the type id, and represent the conversion from blob of memory to object.

[straight-shoota]

But yeah if we want to offer the alignment route, I suppose the pointer needs to be accompanied by a size then? Otherwise the initializer would not know whether there is room for shifting the object?
I suppose this would also allow some flexibility for types with variable size.

[straight-shoota]

Thinking this a bit further, I'm wondering if it wouldn't be more practical to pass an allocator object to the initializer. This encapsulates the decision how much memory even needs to be allocated, and separates it from the mechanism of how it's allocated.

In practice, this could just mean that allocate can receive an optional allocator parameter:

module Allocator
  abstract def self.malloc(size : UInt64)
  abstract def self.malloc_atomic(size : UInt64)
end

class Reference
  module DEFAULT_ALLOCATOR # Maybe this could even point directly to `GC`
    extend Allocator
    def self.malloc(size : UInt64)
      __crystal_malloc64(size)
    end
    def self.malloc_atomic(size : UInt64)
      __crystal_malloc_atomic64(size)
    end
  end

  def self.allocate(allocator : Allocator = DEFAULT_ALLOCATOR) : self
    {% if ... %} # type contains any inner pointers
      obj = allocator.malloc(instance_sizeof(self).to_u64!).as(self)
    {% else %}
      obj = allocator.malloc_atomic(instance_sizeof(self).to_u64!).as(self)
    {% end %}

    Intrinsics.memset(obj.as(Void*), 0_u8, instance_sizeof(self), false)

    {% for ivar in @type.instance_vars %}
      {% if ivar.has_default_value? %}
        pointerof(obj.@{{ ivar }}).value = {{ ivar.default_value }}
      {% end %}
    {% end %}

    set_crystal_type_id(obj)

    obj
  end
end

[HertzDevil]

any other instance where obj is not equal to mem.as(Foo)

An extreme example would be Win32 HGLOBALs; some memory is allocated using GlobalAlloc, whereas pre_initialize calls GlobalLock. Then the two pointers need not be identical if the memory was allocated as movable.

But yeah if we want to offer the alignment route, I suppose the pointer needs to be accompanied by a size then? Otherwise the initializer would not know whether there is room for shifting the object?

You could align a pointer using its address alone. The allocation call is responsible for ensuring enough storage is present after alignment, whether the call happens inside .allocate, or outside it like the HeapAlloc example.

that allocate can receive an optional allocator parameter

This has its own merits but I don't think it affects .pre_initialize's signature, unless the instance_sizeof(self) is passed in as an argument, i.e. we want .pre_initialize to also support String and Log::Metadata. I haven't considered that aspect yet.

[straight-shoota]

I think alignment is deeply connected with the actual memory allocation. So that part of the code should take care of that.
.pre_initialize should only worry about internal setup of the allocated memory.

[ysbaddaden]

I'm wondering if that could allow to 'embed' classes, so that a class could be inlined right into another class? Just like a struct is inlined for example. Yet still be able to refer to the embedded object as normal (@obj is still a reference that can be passed around).

This is a feature I'd love to have to make sure some data is allocated next to each other (less CPU cache trashing) and help reduce HEAP allocations by making a single allocation instead of 1 + N. The drawback being that if all references go out of scope but one reference to an inner object the one big allocation won't be collected by the GC.

Yet, there are great use cases when the ivar is meant to be internally. For example a Mutex to protect an inner data-structure, or the Deque object inside Channel.

[ysbaddaden]

If we use LibC::SizeT[] instead of UInt8[] then it should be aligned by the ABI?