ffi: Initial implementation

[tianxiadys]

All the following texts were translated from Chinese by GPT-4.
If any wording comes across as offensive, please forgive me, it was not my intention.
Thanks to @bengl early work, this PR evolved from his efforts (#46905).

How to use

Currently, FFI does not support all the platforms that Node.js supports.
To avoid compilation failures on unsupported platforms and to prevent confusion for developers,
I have added a compile-time flag, which is disabled by default.
Here is an example of the configure commands:

./configure.py --with-ffi

Or you are on the Windows platform.

./vcbuild.bat ffi

This is an experimental feature.
You must run Node with the --experimental-ffi flag to use it.
Additionally, it is a security-sensitive feature,
so you also need to use the --allow-ffi flag.
By default, all permissions are enabled unless the permission model
is manually activated by using the flag --permission or any --allow-* flag.
Here is a command-line example:

node --experimental-ffi --allow-ffi demo.js

This is a test code that can run on the Windows platform.

const {
  dlopen,
  UnsafeCallback,
  UnsafeFnPointer,
  UnsafePointer,
  UnsafePointerView
} = require('node:ffi');

let a1 = dlopen('user32', {
  EnumWindows: {
    result: 'i32',
    parameters: ['pointer', 'u64']
  }
});
let a2 = new UnsafeCallback({
  result: 'i32',
  parameters: ['pointer', 'u64']
}, (hWnd, lParam) => {
  console.log('hWnd', hWnd, 'lParam', lParam);
  return 1n;
});
a1.symbols.EnumWindows(a2.pointer, 3);
console.log(a1);
console.log(a2);

let b1 = dlopen('kernel32', {
  MultiByteToWideChar: {
    result: 'i32',
    parameters: ['u32', 'u32', 'pointer', 'i32', 'pointer', 'i32']
  },
  HeapAlloc: {
    result: 'pointer',
    parameters: ['pointer', 'u32', 'usize']
  },
  GetProcessHeap: {
    result: 'pointer',
    parameters: []
  }
});
let b2 = Buffer.from('你好', 'utf-8');
let b3 = b1.symbols.GetProcessHeap();
let b4 = b1.symbols.HeapAlloc(b3, 0, 100);
let b5 = b1.symbols.MultiByteToWideChar(65001, 0, b2, -1, b4, 50);
let b6 = UnsafePointerView.getArrayBuffer(b4, 10, 0);
console.log(b1);
console.log(b2);
console.log(b3);
console.log(b4);
console.log(b5);
console.log(b6);

About the "libffi/fixed" and automake dependency

The core of this feature relies on calling the libffi library.
Referencing libffi is challenging because it heavily depends on automake and has poor support for MSVC.
To successfully integrate this library, I made some adjustments.
First, I streamlined the library's files by removing parts we don't currently need.
These can be added back in as needed.
Next, I created a separate fixed folder where I wrote files:
ffi.h, ffi.c, ffiasm.S, fficonfig.h and ffitarget.h.
These files use #if macros to handle most of the work that would normally be done by automake.
Upgrading libffi in the future is both necessary and entirely feasible.
I only added files without modifying any of libffi's existing files.
When updating to a newer version of libffi,
you simply need to replace the corresponding files.

Support Platform

Here is the system and CPU support status:

	Windows	Linux	Mac	Other System
x86	NO	YES		ToDo
x64	YES	YES	YES	ToDo
ARM32	YES	YES		ToDo
ARM64	YES	YES	YES	ToDo
ARM-EC	NO
Other Arch		ToDo		ToDo

1. Blank spaces in the table indicate the absence of such combinations.
   Windows does not support the fifth type of CPU,
   and Mac only supports x64 and ARM64.
2. Windows x86 is the most unique platform because it has numerous calling conventions,
   which are an important part of a function's signature.
   However, our JS API does not have a field to specify the calling convention.
   Fortunately, this platform is no longer actively supported by Node.js,
   so we can reasonably abandon it.
3. The hybrid architecture (Windows ARM-EC) is a crazy idea,
   and I’m not sure whether this technology is widely used.
   Undoubtedly, it presents significant challenges for FFI.
   Currently, it is not supported and likely won’t be in the future either.
4. Support for other architectures and systems will be addressed in future PRs.

About ffi double

For @atgreen:
It seems that a double type field is missing in ffi_raw.
Was this intentionally designed?
Although this issue can be resolved by forcibly casting the pointer type,
I didn't do so because I suspect there might be some traps I'm unaware of waiting for me.

About compatibility with Deno.js

For @aapoalas:
A portion of the content remains unimplemented, divided into three parts:

• the UnsafeCallback.threadSafe method
• the nonblocking option,
• and all methods of UnsafePointerView except static getArrayBuffer.

threadSafe and nonblocking

the threadSafe method should not be able to exist independently of nonblocking.
The most complex callback scenarios I can think of
are the Win32 window procedure callback and APC invocation.
The former involves registering a callback method in RegisterWndClass,
and later, during the message loop,
the thread processes the window procedure by calling DispatchMessage.
The latter involves registering a callback method in ReadFileEx,
and subsequently, the APC callback is entered internally within the SleepEx method.
In any case, the prerequisite for the current thread to enter a callback function
is that the thread must call a certain method,
and this method will internally locate the callback function pointer recorded somewhere,
and then enter the callback function.
A thread cannot arbitrarily enter a callback function at any location;
perhaps only Linux's signal method has such magical capabilities.
In summary, threadSafe seems to apply only to certain specific scenarios.
For example, an independent thread outside of the Node.js event loop executes some kind of loop,
and at a certain point in the loop,
it sends a message to the Node.js main thread.
Wouldn't such a requirement be more suitable for implementation
in a third-party library rather than in the core code of Node.js or Deno.js?

UnsafePointerView

The functionality of UnsafePointerView is to read arbitrary memory addresses in JavaScript.
However, after creating an ArrayBuffer via a pointer,
most of the features of this class become redundant,
as they can be replaced by TypedArray, DataView, or Buffer (Node.js only).
Therefore, UnsafePointerView seems to merely act as a wrapper?

Open issues

I can't wait to share this module with everyone!
The following tasks have not been completed yet,
but perhaps they can be ignored in this PR and completed in future PRs:

1. Documentation for the ffi module (nightly users can refer to Deno.js's documentation).
2. Support only for Windows, Linux, Mac platforms, and certain CPU architectures.
3. Auto-update scripts for libffi.
4. Unit test code and benchmark test code
5. The double type is not supported.
6. Not fully compatible with Deno.js.
7. Other details I haven't thought of yet.

[nodejs-github-bot]

Review requested:

• @nodejs/gyp
• @nodejs/security-wg

[aapoalas]

Hello! Great to see this moving along, and looking forward to collaborating with you on this from my part.

I'll answer your questions / comments as best as I can.

threadSafe and nonblocking

nonblocking in Deno's FFI works by kicking off a separate thread where the actual C call is performed. A Promise is created that resolves when the other thread terminates. This feature could be implemented "natively" with some pain through a Worker. From an implementation point of view, it was deemed both powerful and simple enough to implement as part of the runtime instead of leaving this for a 3rd party library to implement.

UnsafeCallback.threadSafe, on the other hand, is something that I think must be done at the runtime level. The use-case for threadSafe is that a C FFI library may call a given callback from on a thread different from the JavaScript main thread. Of course, we cannot perform the call into V8 on that thread (I have actually made this mistake and in limited cases it even nearly works, but just makes for really weird interactions). So, instead of giving the actual JavaScript callback function pointer to the C FFI library, Deno instead gives a special "synchronization" function. When called, that synchronization function detects if it is running on the main thread or in a foreign thread, and if it runs in a foreign thread it sends a message for the JavaScript main thread / event loop to wake up and blocks. The main thread then wakes up, runs the event loop, sees the message from the foreign thread which also includes pointers to the incoming callback's parameters and return value write location, and performs the actual JavaScript callback with the given parameters. The result is then finally written to the value write location and a message is sent back to the foreign thread for it to stop blocking.

So in short, threadSafe is a synchronizing callback that can be called from any thread, instead of only being called from the JavaScript main thread. Because this synchronization includes needing to wake up the event loop, it requires cooperation from the runtime. There is no way to implement this feature in a 3rd party library.

UnsafePointerView

Yes, the class is somewhat redundant, though in my opinion not entirely.

First some related context: V8 has deprecated TypedArrays and ArrayBuffers in the V8 Fast API for reasons. Deno's FFI implementation relies heavily on the Fast API, making the heavy usage of TypedArrays and ArrayBuffers in the FFI API somewhat problematic and even regrettable. At this point, I personally am of the opinion that the v8::External / UnsafePointer / "pointer" should perhaps be the only supported way to pass pointers in FFI. Similarly, the APIs for creating an ArrayBuffer from a pointer may have been problematic in hindsight.

Admittedly, being able to share memory between JS and C FFI libraries and access that memory though eg. Uint8Array is massively useful. There are times when that might not be entirely correct, though: I don't know if V8 does redundant load removal, but eg. with memory-mapped I/O reading the same index of a Uint8Array twice in a row might look from V8's perspective as redundant, but wouldn't necessarily be if the index maps to a GPIO pin or such. For these cases, the UnsafePointerView would be more appropriate, although a DataView should presumably also perform the same role.

The one API that I very much wouldn't want to lose from UnsafePointerView is the pointer reading API: While it's possible to use reading of a u64 value and UnsafePointer.create(value) to achieve the same, the create API is pretty much the most dangerous API in the entire bunch. If it were possible, I would even remove it entirely from existence. Losing the pointer reading API (and actually we'd also need a pointer writing API) would make create even more important, more commonly used, and thus harder to tell when it gets used. I would not wish that.

[jasnell]

Just a first pass. Spotted a few other concerns but wanted to start with a smaller handful of simple ones.

[jasnell]

Suggested change

	SysIsLE
	SysIsLE,

Code-style is to use trailing commas on these

[jasnell]

Suggested change

	FinalizationRegistry
	FinalizationRegistry,

[jasnell]

Suggested change

	ERR_FFI_UNSUPPORTED_TYPE
	ERR_FFI_UNSUPPORTED_TYPE,

[jasnell]

Suggested change

	getOptionValue
	getOptionValue,

[jasnell]

Suggested change

	setInterval
	setInterval,

[jasnell]

v8::Local cannot be safely allocated on the heap like this. You'll want to use v8::LocalVector instead.

[jasnell]

Use C++ style casts (e.g. static_cast<...>) instead of C style casts

[jasnell]

In header files we use the fully qualified names. This list of using decls needs to be moved into the c++ file

[jasnell]

readAddress(...) can return nullptr. These methods should check for and defend against that.

[atgreen]

libffi's "raw" api was originally developed for the deprecated java APIs used by gcj. It is not widely used. However, there are some performance advantages, for sure. The lack of double in ffi_raw shouldn't be a concern. It's not in there because ffi_raw should fit in the natural register size for the host system, which is not the case for double on many systems.