[vm] Implement Finalizer

This CL implements the `Finalizer` in the GC.

(This CL does not yet implement `NativeFinalizer`.)

The GC is specially aware of two types of objects for the purposes of
running finalizers.

1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)

A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).

A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.

When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.

When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.

To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.

Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.

When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.

Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.

Bug: #47777

TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc

Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>

runtime/docs/gc.md

@@ -13,7 +13,7 @@ A tag of 1 has no penalty on heap object access because removing the tag can be
 Heap objects are always allocated in double-word increments. Objects in old-space are kept at double-word alignment (address % double-word == 0), and objects in new-space are kept offset from double-word alignment (address % double-word == word). This allows checking an object's age without comparing to a boundary address, avoiding restrictions on heap placement and avoiding loading the boundary from thread-local storage. Additionally, the scavenger can quickly skip over both immediates and old objects with a single branch.
 
 | Pointer    | Referent                                |
-| ---        | ---                                     |
+| ---------- | --------------------------------------- |
 | 0x00000002 | Small integer 1                         |
 | 0xFFFFFFFE | Small integer -1                        |
 | 0x00A00001 | Heap object at 0x00A00000, in old-space |
@@ -75,7 +75,7 @@ The Dart VM includes a sliding compactor. The forwarding table is compactly repr
 
 ## Concurrent Marking
 
-To reduce the time the mutator is paused for old-space GCs, we allow the mutator to continue running during most of the marking work. 
+To reduce the time the mutator is paused for old-space GCs, we allow the mutator to continue running during most of the marking work.
 
 ### Barrier
 
@@ -204,3 +204,35 @@ container <- AllocateObject
 <instructions that cannot directly call Dart functions>
 StoreInstanceField(container, value, NoBarrier)
 ```
+
+## Finalizers
+
+The GC is aware of two types of objects for the purposes of running finalizers.
+
+1) `FinalizerEntry`
+2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
+
+A `FinalizerEntry` contains the `value`, the optional `detach` key, and the `token`, and a reference to the `finalizer`.
+An entry only holds on weakly to the value, detach key, and finalizer. (Similar to how `WeakReference` only holds on weakly to target).
+
+A `Finalizer` contains all entries, a list of entries of which the value is collected, and a reference to the isolate.
+
+When the value of an entry is GCed, the entry is added over to the collected list.
+If any entry is moved to the collected list, a message is sent that invokes the finalizer to call the callback on all entries in that list.
+
+When a finalizer is detached by the user, the entry token is set to the entry itself and is removed from the all entries set.
+This ensures that if the entry was already moved to the collected list, the finalizer is not executed.
+
+To speed up detaching, we use a weak map from detach keys to list of entries. This ensures entries can be GCed.
+
+Both the scavenger and marker can process finalizer entries in parallel.
+Parallel tasks use an atomic exchange on the head of the collected entries list, ensuring no entries get lost.
+Mutator threads are guaranteed to be stopped when processing entries.
+This ensures that we do not need barriers for moving entries into the finalizers collected list.
+Dart reads and replaces the collected entries list also with an atomic exchange, ensuring the GC doesn't run in between a load/store.
+
+When a finalizer gets posted a message to process finalized objects, it is being kept alive by the message.
+An alternative design would be to pre-allocate a `WeakReference` in the finalizer pointing to the finalizer, and send that itself.
+This would be at the cost of an extra object.
+
+If the finalizer object itself is GCed, the callback is not run for any of the attachments.

runtime/lib/isolate.cc

@@ -242,6 +242,8 @@ static ObjectPtr ValidateMessageObject(Zone* zone,
     break;
 
           MESSAGE_SNAPSHOT_ILLEGAL(DynamicLibrary);
+          // TODO(http://dartbug.com/47777): Send and exit support: remove this.
+          MESSAGE_SNAPSHOT_ILLEGAL(Finalizer);
           MESSAGE_SNAPSHOT_ILLEGAL(MirrorReference);
           MESSAGE_SNAPSHOT_ILLEGAL(Pointer);
           MESSAGE_SNAPSHOT_ILLEGAL(ReceivePort);
@@ -284,6 +286,7 @@ static ObjectPtr ValidateMessageObject(Zone* zone,
   return obj.ptr();
 }
 
+// TODO(http://dartbug.com/47777): Add support for Finalizers.
 DEFINE_NATIVE_ENTRY(Isolate_exit_, 0, 2) {
   GET_NATIVE_ARGUMENT(SendPort, port, arguments->NativeArgAt(0));
   if (!port.IsNull()) {
@@ -638,9 +641,10 @@ class SpawnIsolateTask : public ThreadPool::Task {
     // Make a copy of the state's isolate flags and hand it to the callback.
     Dart_IsolateFlags api_flags = *(state_->isolate_flags());
     api_flags.is_system_isolate = false;
-    Dart_Isolate isolate = (create_group_callback)(
-        state_->script_url(), name, nullptr, state_->package_config(),
-        &api_flags, parent_isolate_->init_callback_data(), &error);
+    Dart_Isolate isolate =
+        (create_group_callback)(state_->script_url(), name, nullptr,
+                                state_->package_config(), &api_flags,
+                                parent_isolate_->init_callback_data(), &error);
     parent_isolate_->DecrementSpawnCount();
     parent_isolate_ = nullptr;
 

runtime/tests/vm/dart/finalizer/finalizer_isolate_groups_run_gc_helper.dart

@@ -0,0 +1,36 @@
+// Copyright (c) 2022, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'dart:isolate';
+
+import 'helpers.dart';
+
+int callbackCount = 0;
+
+void callback(Nonce token) {
+  callbackCount++;
+  print('$name: Running finalizer: token: $token');
+}
+
+final finalizer = Finalizer<Nonce>(callback);
+
+late String name;
+
+void main(List<String> arguments, SendPort port) async {
+  name = arguments[0];
+
+  final token = Nonce(42);
+  makeObjectWithFinalizer(finalizer, token);
+
+  final awaitBeforeShuttingDown = ReceivePort();
+  port.send(awaitBeforeShuttingDown.sendPort);
+  final message = await awaitBeforeShuttingDown.first;
+  print('$name: $message');
+
+  await Future.delayed(Duration(milliseconds: 1));
+  print('$name: Awaited to see if there were any callbacks.');
+
+  print('$name: Helper isolate exiting. num callbacks: $callbackCount.');
+  port.send(callbackCount);
+}

runtime/tests/vm/dart/finalizer/finalizer_isolate_groups_run_gc_test.dart

@@ -0,0 +1,93 @@
+// Copyright (c) 2022, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+// VMOptions=
+// VMOptions=--use_compactor
+// VMOptions=--use_compactor --force_evacuation
+
+import 'dart:async';
+import 'dart:isolate';
+
+import 'package:async/async.dart';
+import 'package:expect/expect.dart';
+
+import 'helpers.dart';
+
+void main() async {
+  await testFinalizerInOtherIsolateGroupGCBeforeExit();
+  await testFinalizerInOtherIsolateGroupGCAfterExit();
+  await testFinalizerInOtherIsolateGroupNoGC();
+
+  print('$name: End of test, shutting down.');
+}
+
+const name = 'main';
+
+late bool hotReloadBot;
+
+Future<void> testFinalizerInOtherIsolateGroupGCBeforeExit() async {
+  final receivePort = ReceivePort();
+  final messagesQueue = StreamQueue(receivePort);
+
+  await Isolate.spawnUri(
+    Uri.parse('finalizer_isolate_groups_run_gc_helper.dart'),
+    ['helper 1'],
+    receivePort.sendPort,
+  );
+  final signalHelperIsolate = await messagesQueue.next as SendPort;
+
+  doGC(name: name);
+  await yieldToMessageLoop(name: name);
+
+  signalHelperIsolate.send('Done GCing.');
+
+  final helperCallbacks = await messagesQueue.next as int;
+  messagesQueue.cancel();
+  print('$name: Helper exited.');
+  // Different isolate group, so we don't expect a GC in this isolate to cause
+  // collected objects in the helper.
+  // Except for in --hot-reload-test-mode, then the GC is triggered.
+  hotReloadBot = helperCallbacks == 1;
+}
+
+Future<void> testFinalizerInOtherIsolateGroupGCAfterExit() async {
+  final receivePort = ReceivePort();
+  final messagesQueue = StreamQueue(receivePort);
+  await Isolate.spawnUri(
+    Uri.parse('finalizer_isolate_groups_run_gc_helper.dart'),
+    ['helper 2'],
+    receivePort.sendPort,
+  );
+
+  final signalHelperIsolate = await messagesQueue.next as SendPort;
+
+  signalHelperIsolate.send('Before GCing.');
+
+  final helperCallbacks = await messagesQueue.next as int;
+  messagesQueue.cancel();
+  print('$name: Helper exited.');
+  Expect.equals(hotReloadBot ? 1 : 0, helperCallbacks);
+
+  doGC(name: name);
+  await yieldToMessageLoop(name: name);
+}
+
+Future<void> testFinalizerInOtherIsolateGroupNoGC() async {
+  final receivePort = ReceivePort();
+  final messagesQueue = StreamQueue(receivePort);
+
+  await Isolate.spawnUri(
+    Uri.parse('finalizer_isolate_groups_run_gc_helper.dart'),
+    ['helper 3'],
+    receivePort.sendPort,
+  );
+  final signalHelperIsolate = await messagesQueue.next as SendPort;
+
+  signalHelperIsolate.send('Before quitting main isolate.');
+
+  final helperCallbacks = await messagesQueue.next as int;
+  messagesQueue.cancel();
+  print('$name: Helper exited.');
+  Expect.equals(hotReloadBot ? 1 : 0, helperCallbacks);
+}

runtime/tests/vm/dart/finalizer/finalizer_isolates_run_gc_test.dart

@@ -0,0 +1,103 @@
+// Copyright (c) 2022, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+// VMOptions=
+// VMOptions=--use_compactor
+// VMOptions=--use_compactor --force_evacuation
+
+import 'dart:async';
+import 'dart:isolate';
+
+import 'package:expect/expect.dart';
+
+import 'helpers.dart';
+
+void main() async {
+  await testNormalExit();
+  await testSendAndExit();
+  await testSendAndExitFinalizer();
+  print('End of test, shutting down.');
+}
+
+final finalizerTokens = <Nonce>{};
+
+void callback(Nonce token) {
+  print('Running finalizer: token: $token');
+  finalizerTokens.add(token);
+}
+
+void runIsolateAttachFinalizer(Object? message) {
+  final finalizer = Finalizer<Nonce>(callback);
+  final value = Nonce(1001);
+  final token = Nonce(1002);
+  finalizer.attach(value, token);
+  final token9 = Nonce(9002);
+  makeObjectWithFinalizer(finalizer, token9);
+  if (message == null) {
+    print('Isolate done.');
+    return;
+  }
+  final list = message as List;
+  assert(list.length == 2);
+  final sendPort = list[0] as SendPort;
+  final tryToSendFinalizer = list[1] as bool;
+  if (tryToSendFinalizer) {
+    Expect.throws(() {
+      // TODO(http://dartbug.com/47777): Send and exit support.
+      print('Trying to send and exit finalizer.');
+      Isolate.exit(sendPort, [value, finalizer]);
+    });
+  }
+  print('Isolate sending and exit.');
+  Isolate.exit(sendPort, [value]);
+}
+
+Future testNormalExit() async {
+  final portExitMessage = ReceivePort();
+  await Isolate.spawn(
+    runIsolateAttachFinalizer,
+    null,
+    onExit: portExitMessage.sendPort,
+  );
+  await portExitMessage.first;
+
+  doGC();
+  await yieldToMessageLoop();
+
+  Expect.equals(0, finalizerTokens.length);
+}
+
+@pragma('vm:never-inline')
+Future<Finalizer?> testSendAndExitHelper(
+    {bool trySendFinalizer = false}) async {
+  final port = ReceivePort();
+  await Isolate.spawn(
+    runIsolateAttachFinalizer,
+    [port.sendPort, trySendFinalizer],
+  );
+  final message = await port.first as List;
+  print('Received message ($message).');
+  final value = message[0] as Nonce;
+  print('Received value ($value), but now forgetting about it.');
+
+  Expect.equals(1, message.length);
+  // TODO(http://dartbug.com/47777): Send and exit support.
+  return null;
+}
+
+Future testSendAndExit() async {
+  await testSendAndExitHelper(trySendFinalizer: false);
+
+  doGC();
+  await yieldToMessageLoop();
+
+  Expect.equals(0, finalizerTokens.length);
+}
+
+Future testSendAndExitFinalizer() async {
+  final finalizer = await testSendAndExitHelper(trySendFinalizer: true);
+
+  // TODO(http://dartbug.com/47777): Send and exit support.
+  Expect.isNull(finalizer);
+}

runtime/tests/vm/dart/finalizer/finalizer_nullable_run_gc_test.dart

@@ -0,0 +1,43 @@
+// Copyright (c) 2022, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'package:expect/expect.dart';
+
+import 'helpers.dart';
+
+void main() {
+  testFinalizer();
+}
+
+void testFinalizer() async {
+  final finalizerTokens = <Nonce?>{};
+  void callback(Nonce? token) {
+    print('Running finalizer: token: $token');
+    finalizerTokens.add(token);
+  }
+
+  final finalizer = Finalizer<Nonce?>(callback);
+
+  {
+    final detach = Nonce(2022);
+    final token = null;
+
+    makeObjectWithFinalizer(finalizer, token, detach: detach);
+
+    doGC();
+
+    // We haven't stopped running synchronous dart code yet.
+    Expect.isFalse(finalizerTokens.contains(token));
+
+    await Future.delayed(Duration(milliseconds: 1));
+
+    // Now we have.
+    Expect.isTrue(finalizerTokens.contains(token));
+
+    // Try detaching after finalizer ran.
+    finalizer.detach(detach);
+  }
+
+  print('End of test, shutting down.');
+}