Calling exit() after OutOfMemoryError does not exit program

[feli-citas]

Trying to exit a Dart program after catching an OutOfMemoryError leads to the following behavior (call to exit happens right after printing oom):

out/DebugIA32/dart --enable-interpreter --optimization_level=3 --old_gen_heap_size=128 out.dill                      
foo0 throws
X0().foo0_1() throws
[9223372032559808639, 91, -58, -37, -39, 82, -45]
Exhausted heap space, trying to allocate 16 bytes.
oom
Exhausted heap space, trying to allocate 24 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 16 bytes.
...

To reproduce:
fuzz.dart.txt

sdk/pkg/vm/tool/gen_kernel --gen-bytecode --platform=sdk/out/DebugIA32/vm_platform_strong.dill -o out.dill fuzz.dart.txt
sdk/out/DebugIA32/dart --enable-interpreter --optimization_level=3 --old_gen_heap_size=128 out.dill

OutOfMemoryError is thrown in X2.foo2_0().

[feli-citas]

Isolate (/b/s/w/itIA9Uyn/dart_fuzzDOAFVW) NO-FFI NO-FP : KBC-INT-noVFP-ReleaseSIMARM - KBC-INT-DebugX64: !DIVERGENCE! 1.53:1938374226 (0 vs -6)

fail2:
-6
foo1 throws
foo2 throws
foo3 throws
X1().foo1_1() throws                                                                                                                                                                                                                                                                      


  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
oom
  
Exhausted heap space, trying to allocate 64 bytes.
Exhausted heap space, trying to allocate 64 bytes.
Exhausted heap space, trying to allocate 96 bytes.
../../runtime/vm/object.cc: 2214: error: Out of memory.
version=2.6.0-edge.e4d8ad5866ec948c7b8be8d8ab1e54b6b263984b (Sat Sep 28 01:55:45 2019 +0000) on "linux_x64"
thread=4193, isolate=main(0x5613ff989800)
  pc 0x00005613fd4b8f3c fp 0x00007f56f163d150 dart::Profiler::DumpStackTrace(void*)
  pc 0x00005613fd0de952 fp 0x00007f56f163d230 dart::Assert::Fail(char const*, ...)
  pc 0x00005613fd3ea808 fp 0x00007f56f163d2a0 dart::Object::Allocate(long, long, dart::Heap::Space)
  pc 0x00005613fd45aa55 fp 0x00007f56f163d310 dart::Bytecode::New(unsigned long, long, long, dart::ObjectPool const&)
  pc 0x00005613fd6dba63 fp 0x00007f56f163d3b0 dart::kernel::BytecodeReaderHelper::ReadBytecode(dart::ObjectPool const&)
  pc 0x00005613fd6da272 fp 0x00007f56f163d470 dart::kernel::BytecodeReaderHelper::ReadCode(dart::Function const&, long)
  pc 0x00005613fd6e5f69 fp 0x00007f56f163d870 dart::kernel::BytecodeReader::ReadFunctionBytecode(dart::Thread*, dart::Function const&)
  pc 0x00005613fd34cb20 fp 0x00007f56f163d910 dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  pc 0x00005613fd34ec59 fp 0x00007f56f163d950 dart::DartLibraryCalls::ToString(dart::Instance const&)
  pc 0x00005613fd39f727 fp 0x00007f56f163d9e0 dart::IsolateMessageHandler::ProcessUnhandledException(dart::Error const&)
  pc 0x00005613fd39ee81 fp 0x00007f56f163dbe0 dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message> >)
  pc 0x00005613fd3decce fp 0x00007f56f163dc70 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  pc 0x00005613fd3dfc16 fp 0x00007f56f163dcf0 dart::MessageHandler::TaskCallback()
  pc 0x00005613fd56db1c fp 0x00007f56f163dd40 dart::ThreadPool::Worker::Loop()
  pc 0x00005613fd56d604 fp 0x00007f56f163dd90 dart::ThreadPool::Worker::Main(unsigned long)
  pc 0x00005613fd4b2e15 fp 0x00007f56f163ded0 /b/s/w/ir/out/DebugX64/dart+0x17d4e15
-- End of DumpStackTrace



-- BEGIN REPRODUCE  --

dartfuzz.dart --no-ffi --no-fp --seed 1938374226 /b/s/w/itIA9Uyn/dart_fuzzDOAFVW/fuzz.dart

-- RUN 1 --

/b/s/w/ir/pkg/vm/tool/gen_kernel --gen-bytecode --platform=/b/s/w/ir/out/ReleaseSIMARM/vm_platform_strong.dill -o /b/s/w/itIA9Uyn/dart_fuzzDOAFVW/out.dill /b/s/w/itIA9Uyn/dart_fuzzDOAFVW/fuzz.dart
/b/s/w/ir/out/ReleaseSIMARM/dart --enable-interpreter --compilation-counter-threshold=-1 --no-use-vfp --old_gen_heap_size=128 /b/s/w/itIA9Uyn/dart_fuzzDOAFVW/out.dill

-- RUN 2 --

/b/s/w/ir/pkg/vm/tool/gen_kernel --gen-bytecode --platform=/b/s/w/ir/out/DebugX64/vm_platform_strong.dill -o /b/s/w/itIA9Uyn/dart_fuzzDOAFVW/out.dill /b/s/w/itIA9Uyn/dart_fuzzDOAFVW/fuzz.dart
/b/s/w/ir/out/DebugX64/dart --enable-interpreter --compilation-counter-threshold=-1 --old_gen_heap_size=128 /b/s/w/itIA9Uyn/dart_fuzzDOAFVW/out.dill

-- END REPRODUCE  --

Isolate (/b/s/w/itu8EJKS/dart_fuzzZDNHRH) NO-FFI FP : JIT-O3-SLOWPATH-ReleaseSIMARM - KBC-MIX-O3-DebugIA32: !DIVERGENCE! 1.53:1570145583 (0 vs -6)

fail2:
-6
foo0 throws
X0().foo0_1() throws
[9223372032559808639, 91, -58, -37, -39, 82, -45]
oom

Exhausted heap space, trying to allocate 16 bytes.
Exhausted heap space, trying to allocate 40 bytes.
../../runtime/vm/object.cc: 2214: error: Out of memory.
version=2.6.0-edge.e4d8ad5866ec948c7b8be8d8ab1e54b6b263984b (Sat Sep 28 01:55:45 2019 +0000) on "linux_ia32"
thread=17831, isolate=main(0x4b66700)
  pc 0x0198471c fp 0xf703e8e8 dart::Profiler::DumpStackTrace(void*)
  pc 0x01e8f1f4 fp 0xf703e908 Dart_DumpNativeStackTrace
  pc 0x015fc17b fp 0xf703e938 dart::Assert::Fail(char const*, ...)
  pc 0x018c1b2d fp 0xf703e998 dart::Object::Allocate(int, int, dart::Heap::Space)
  pc 0x01945b34 fp 0xf703e9e8 dart::Array::New(int, int, dart::Heap::Space)
  pc 0x018c8856 fp 0xf703ea18 dart::Array::New(int, dart::Heap::Space)
  pc 0x01a8b423 fp 0xf703ea78 dart::FlowGraphCompiler::InitCompiler()
  pc 0x01a984a5 fp 0xf703eab8 dart::FlowGraphCompiler::CompileGraph()
  pc 0x01bdc208 fp 0xf703ee48 dart::CompileParsedFunctionHelper::Compile(dart::CompilationPipeline*)
  pc 0x01bdcf95 fp 0xf703efd8 /b/s/w/ir/out/DebugIA32/dart+0x17dcf95
  pc 0x01bdca60 fp 0xf703f058 dart::Compiler::CompileFunction(dart::Thread*, dart::Function const&)
  pc 0x01bdf191 fp 0xf703f0d8 dart::BackgroundCompiler::Run()
  pc 0x01be0307 fp 0xf703f0f8 /b/s/w/ir/out/DebugIA32/dart+0x17e0307
  pc 0x01a2d7ff fp 0xf703f148 dart::ThreadPool::Worker::Loop()
  pc 0x01a2d39a fp 0xf703f198 dart::ThreadPool::Worker::Main(unsigned int)
  pc 0x0197ea1f fp 0xf703f2e8 /b/s/w/ir/out/DebugIA32/dart+0x157ea1f
  pc 0xf7c9d295 fp 0xf703f3a8 /lib/i386-linux-gnu/libpthread.so.0+0x6295
-- End of DumpStackTrace
  


-- BEGIN REPRODUCE  --

dartfuzz.dart --no-ffi --fp --seed 1570145583 /b/s/w/itu8EJKS/dart_fuzzZDNHRH/fuzz.dart

-- RUN 1 --

/b/s/w/ir/out/ReleaseSIMARM/dart --optimization_level=3 --use-slow-path --old_gen_heap_size=128 /b/s/w/itu8EJKS/dart_fuzzZDNHRH/fuzz.dart

-- RUN 2 --

/b/s/w/ir/pkg/vm/tool/gen_kernel --gen-bytecode --platform=/b/s/w/ir/out/DebugIA32/vm_platform_strong.dill -o /b/s/w/itu8EJKS/dart_fuzzZDNHRH/out.dill /b/s/w/itu8EJKS/dart_fuzzZDNHRH/fuzz.dart
/b/s/w/ir/out/DebugIA32/dart --enable-interpreter --optimization_level=3 --old_gen_heap_size=128 /b/s/w/itu8EJKS/dart_fuzzZDNHRH/out.dill

-- END REPRODUCE  --

[alexmarkov]

These fatal Out of memory errors come from Object::Allocate when there are no Dart frames on the stack:

sdk/runtime/vm/object.cc

Lines 2205 to 2215 in 692230b

	if (thread->top_exit_frame_info() != 0) {
	// Use the preallocated out of memory exception to avoid calling
	// into dart code or allocating any code.
	const Instance& exception =
	Instance::Handle(thread->isolate()->object_store()->out_of_memory());
	Exceptions::Throw(thread, exception);
	UNREACHABLE();
	} else {
	// No Dart to propagate an exception to.
	OUT_OF_MEMORY();
	}

It's not clear what would be the best way to handle out of memory in this case. It might be valid to throw Dart exception in these cases (there are LongJumpScope to catch them both in bytecode reader and compiler), but I'm not sure the exception would be properly handled and shown to user.

In the first case, OOM is triggered by bytecode reading in an attempt to execute some Dart code to show the exception from IsolateMessageHandler::ProcessUnhandledException. Maybe IsolateMessageHandler::ProcessUnhandledException should recognize out of memory exception and not even try to run any Dart code in such case.

The second stack trace is for OOM in background compiler. While we can silently abort background compilation and not show anything, it might hide the underlying problem from users.

/cc @rmacnak-google @a-siva

[aartbik]

We print less and exit earlier on OOM, with false divergence detection at test runner level, and also are avoiding OOM more and more. So I am closing this bug since it no longer seems blocking DartFuzz.