Clearing memory securely

[HertzDevil]

In security-sensitive applications, it is often necessary to clear some memory buffer with zeros as soon as the program has finished using it. However, compiler optimizations may treat certain clears as dead stores and remove them in release builds, meaning the buffer might be leaked:

def foo(user)
  password = UInt8.static_array(0x61, 0x64, 0x6d, 0x69, 0x6e)

  success = login(user, password)

  # `password` is on the stack and can never be accessed afterwards
  # within `#foo`, so this is a dead store;
  # an optimizer might remove this call entirely and
  # leak `password` on the call stack
  password.fill(0)

  success
end

This seems to be such an important issue that even C added memset_explicit to C23 as an alternative to the optional memset_s. Fortunately, the standard library already provides the means to do this correctly:

def foo(user)
  # ...

  # use a volatile store
  Intrinsics.memset(password.to_unsafe, 0, sizeof(typeof(password)), true)

  true
end

To see this in action, observe how foo places a movl $0, 4(%rsp) instruction and foo_insecure doesn't. The standard library interface for this could be as simple as:

struct Pointer(T)
  # I believe this is better than adding an optional parameter to `#clear`, as
  # I don't think there are any use cases where a clear needs to be conditionally (in)secure
  def secure_clear(count = 1)
    Intrinsics.memset(self.as(Void*), 0_u8, bytesize(count), true)
  end
end

Now the buffer can be cleared using password.to_unsafe.secure_clear(password.size), but this is still too much to type, and also probably shouldn't involve any unsafe operations. So perhaps some collections could support this directly:

struct Slice(T)
  def secure_clear
    check_writable
    to_unsafe.secure_clear(size)
  end
end

struct StaticArray(T, N)
  delegate secure_clear, to: to_slice
end

Now this faces a different issue where all zeros might not be a possible representation of T at all, e.g. clearing a Slice(Int32 | Int64) is undefined behavior. So Slice(T)#secure_clear might only be callable when T is a primitive number or pointer type.

The story is a bit different for the dynamic heap. On one hand, a volatile store isn't required because those heap addresses won't become dead in snippets like the above; on the other hand, it is undefined whether LibC.free or GC'ed deallocation zeros the memory, so explicit deallocation might not prevent the buffer from being leaked, thus either Pointer#clear or Pointer#secure_clear is necessary. Maybe this is how a String or an owning Slice can be cleared:

# very unsafe! clears the type ID! `String` is not even supposed to be mutable
# also most other types would simply use `instance_sizeof(typeof(password))` instead
password = String.build &.<< "admin"
password.as(Void*).secure_clear(String::HEADER_SIZE + password.bytesize + 1)

# will actually be the same whether a `Slice` points at stack or heap memory
password = Bytes[0x61, 0x64, 0x6d, 0x69, 0x6e]
password.secure_clear

It looks difficult to define a "safe" API here.

Related: #10764

[straight-shoota]

So Slice(T)#secure_clear might only be callable when T is a primitive number or pointer type.

Sounds like a resonable restriction. I suppose it should perhaps apply to the existing Pointer#clear as well?

[ysbaddaden]

Very nice!

What if T responds to .zero? Could we use this information to clear the buffer? Granted it wouldn't be a single byte, so it might be more complex than just calling memset.

[HertzDevil]

BigInt.zero is not zero-initialized:

BigInt.zero # => LibGMP::MPZ(@_mp_alloc=1, @_mp_size=0, @_mp_d=Pointer(UInt64)@0x7fac878d5ff0) 

[ysbaddaden]

Indeed, but it's still a "zero" value, so we could set it... but then we can't set the volatile flag to tell LLVM to not optimize (got it). I wonder if we could tell LLVM to not optimize a loop/set 🤔