-
-
Notifications
You must be signed in to change notification settings - Fork 32k
node-api: add support for Float16Array #58879
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Review requested:
|
48a7303
to
3d5bf56
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #58879 +/- ##
==========================================
- Coverage 90.09% 90.09% -0.01%
==========================================
Files 640 640
Lines 188491 188433 -58
Branches 36969 36962 -7
==========================================
- Hits 169823 169767 -56
+ Misses 11399 11379 -20
- Partials 7269 7287 +18
🚀 New features to boost your workflow:
|
const arrayTypes = [ Int8Array, Uint8Array, Uint8ClampedArray, Int16Array, | ||
Uint16Array, Int32Array, Uint32Array, Float32Array, | ||
Float64Array, BigInt64Array, BigUint64Array ]; | ||
Float64Array, BigInt64Array, BigUint64Array, Float16Array ]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit:
I think it would be better to add it before Float32Array
for consistency.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@VoltrexKeyva, we must never add new enum entries in Node-API in the middle. It breaks the ABI compatibility rules.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, didn't know that, these suggestions should be reverted then.
const nonByteArrayTypes = [ Int16Array, Uint16Array, Int32Array, Uint32Array, | ||
Float32Array, Float64Array, | ||
BigInt64Array, BigUint64Array ]; | ||
BigInt64Array, BigUint64Array, Float16Array ]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
doc/api/n-api.md
Outdated
@@ -2267,6 +2274,7 @@ typedef enum { | |||
napi_float64_array, | |||
napi_bigint64_array, | |||
napi_biguint64_array, | |||
napi_float16_array |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
@@ -105,6 +105,7 @@ typedef enum { | |||
napi_float64_array, | |||
napi_bigint64_array, | |||
napi_biguint64_array, | |||
napi_float16_array, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
src/js_native_api_v8.cc
Outdated
@@ -3211,6 +3215,8 @@ napi_status NAPI_CDECL napi_get_typedarray_info(napi_env env, | |||
*type = napi_bigint64_array; | |||
} else if (value->IsBigUint64Array()) { | |||
*type = napi_biguint64_array; | |||
} else if (value->IsFloat16Array()) { | |||
*type = napi_float16_array; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
src/js_native_api_types.h
Outdated
@@ -101,6 +101,7 @@ typedef enum { | |||
napi_uint16_array, | |||
napi_int32_array, | |||
napi_uint32_array, | |||
napi_float16_array, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is important to note that Node-API is an ABI stable API. It means that any code that was compiled before with the previous version of Node-API must work in the future version of Node-API. See more details here: https://github.com/nodejs/node/blob/main/doc/contributing/adding-new-napi-api.md
By inserting a new enum value in the middle of the existing enum values we effectively change integer values for all enum values that follow the inserted entry. Any previously compiled code that used to create float32 array will suddenly create the float16 array, float64 became float32, etc.
The only valid place to insert new enum values in Node-API is at the end of an enum.
Also, any Node-API changes must be versioned and initially be part of the experimental version.
I.e. the new enum entry must be under #ifdef NAPI_EXPERIMENTAL
. E.g. see this commit where the new Node-API property attributes were added.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@IlyasShabi , thank you for fixing the enum order!
Like @legendecas mentioned here #58889 (comment), we also need a new experiemental feature macro to be defined. As an example, please see this commit: 59e7444 and how it defined the NODE_API_EXPERIMENTAL_HAS_PROPERTY_KEYS
macro. The macro will be deleted when the code becomes part of an official Node-API version after some "baking" time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to ensure that the new Node-API changes follow the Node-API change requirements outlined in this document: https://github.com/nodejs/node/blob/main/doc/contributing/adding-new-napi-api.md
Note that the requirement that says "Should be implemented in at least one other VM implementation of Node.js." - it is currently being discussed to be revised towards something like "being implementable by other JS runtimes and JS engines" - see #58852. I.e. the new API like in this PR must be good as long as it is not V8-specific.
ABI breaking changes made in subsequent commits. Removing the approval until all issues can be resolved
@IlyasShabi ... to address the ABI breaking issues, you might take some inspiration from this duplicate PR #58889 |
@@ -548,6 +548,7 @@ node_api_post_finalizer(node_api_basic_env env, | |||
void* finalize_data, | |||
void* finalize_hint); | |||
|
|||
#define NODE_API_EXPERIMENTAL_HAS_FLOAT16_ARRAY |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is rather unexpected to see this macro defined here. It must be defined in the same place where you define the new enum value. E.g:
#ifdef NAPI_EXPERIMENTAL
#define NODE_API_EXPERIMENTAL_HAS_FLOAT16_ARRAY
napi_float16_array,
#endif // NAPI_EXPERIMENTAL
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feels quite silly to put this behind an experimental flag – this seems more of a "fix" to account for the fact that the language now exposes a new variant, rather than a new feature.
@@ -3165,6 +3165,13 @@ napi_status NAPI_CDECL napi_create_typedarray(napi_env env, | |||
CREATE_TYPED_ARRAY( | |||
env, BigUint64Array, 8, buffer, byte_offset, length, typedArray); | |||
break; | |||
case napi_float16_array: | |||
if (env->module_api_version != NAPI_VERSION_EXPERIMENTAL) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This bit in particular is odd – why the check here? Even if this feature is experimental, the enum value being used at all should be enough to imply that the caller has opted into this feature.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, it may look silly, but if we want to ensure predictable behavior between Node-API versions, then we must do it.
If we only focus on Node.js, and do not care much about versions, then you are right - it may be a bit of overhead.
But when you start to take into account support for other runtimes and JS engine, having a consistent and well-defined behavior becomes very important. To me this strict well-defined Node-API versioning is one of its best-selling points.
Say, someone creates a plugin that relies on the new Float16Array.
If we add it to Node.js directly without Node-API version, then this addon may work or not work in Deno or Bun depending on stars.
But if we explicitly say that Float16Array is the part of the experimental, and then version 11, then the Deno and Bun developers will be able to provide exactly the same contract and thus give the addon developers the best predictable experience.
Having the predictable behavior in this code file is also important for unit tests. Without versioning here, we cannot create reliable tests that can check the differences between versions. E.g. previously the unit test was successful even if the code relied on the experimental feature. Now it fails as expected because the addon must not use it yet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
depending on stars.
It depends on a very specific question though, not stars; namely, whether the runtime has support for creating Float16Array instances through Node-API. The behavior is predictable; either napi_create_typedarray()
succeeds or it fails, and the caller should be able to handle both.
In any case, do what you must.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! We just follow the established process here.
We often see some completely unexpected side effects even from something that looks quite innocent.
Thus, it is better to be "defensive" about all new features and follow the "book". :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't feel strongly about adding a runtime version guard. IMO, adding an NAPI_EXPERIMENTAL
and NODE_API_EXPERIMENTAL_HAS_FLOAT16_ARRAY
around the definition should be sufficient, and addons can use NODE_API_EXPERIMENTAL_HAS_FLOAT16_ARRAY
to guard if napi_float16_array
is defined.
We added runtime version checks for features like finalizers, which can cause side effects. But for napi_create_typedarray
, I think it should not cause side-effects.
@@ -41,8 +41,8 @@ assert.strictEqual(externalResult[2], 2); | |||
// Validate creation of all kinds of TypedArrays | |||
const buffer = new ArrayBuffer(128); | |||
const arrayTypes = [ Int8Array, Uint8Array, Uint8ClampedArray, Int16Array, | |||
Uint16Array, Int32Array, Uint32Array, Float32Array, | |||
Float64Array, BigInt64Array, BigUint64Array ]; | |||
Uint16Array, Int32Array, Uint32Array, Float16Array, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that the unit test is failing because the test binding.gyp
must be changed to use the NAPI_EXPERIMENTAL
macro. Otherwise, it is compiled with the latest stable Node-API version.
See:
"defines": [ "NAPI_EXPERIMENTAL" ], |
This PR aims to add support for Float16Array in n-api, addressing this issue
cc: @nodejs/node-api