Intent to implement: WebIDL Binding Generator

[romandev]

Hi all,

I'm Jinho Bang and a contributor in this project.

Today, I'd like to suggest a new feature a.k.a WebIDL Binding Generator. If we use it, we can just generate N-API binding code automatically. I've been implementing POC demo with @yjaeseok and @nadongguri for last few weeks.

Let's look deep into WebIDL Binding Generator.

Introduction

To help you better understand, let's see the following example. I did copy N paste an example from abi-stable-node-addon-api examples repo. As you already know, if we use it, we can invoke native add() function in JS side.

Example: add() function

#include <napi.h>

Napi::Value Add(const Napi::CallbackInfo& info) {
  Napi::Env env = info.Env();

  if (info.Length() < 2) {
    Napi::TypeError::New(env, "Wrong number of arguments").ThrowAsJavaScriptException();
    return env.Null();
  }

  if (!info[0].IsNumber() || !info[1].IsNumber()) {
    Napi::TypeError::New(env, "Wrong arguments").ThrowAsJavaScriptException();
    return env.Null();
  }

  double arg0 = info[0].As<Napi::Number>().DoubleValue();
  double arg1 = info[1].As<Napi::Number>().DoubleValue();
  Napi::Number num = Napi::Number::New(env, arg0 + arg1);

  return num;
}

Napi::Object Init(Napi::Env env, Napi::Object exports) {
  exports.Set(Napi::String::New(env, "add"),
              Napi::Function::New(env, Add));
  return exports;
}

NODE_API_MODULE(addon, Init)

The above example is not difficult but it requires developer write more code than they think. If we could simplify the above example as follows, we will reduce the developer's plumbing.

double add(double x, double y) {
  return x + y;
}

Isn't this just native code? Yes! So, how can Javascript call the native code? The key solution is WebIDL. We can make Javascript call the native code internally just by writing WebIDL like this:

interface Calculator {
  double add(double x, double y);
};

What is the WebIDL Binding Generator?

The WebIDL Binding Generator is based on WebIDL to generate a binding code automatically.

WebIDL Binding Generator is typically used in the Chromium project. Chromium uses the Blink engine and the Javascript V8 engine. They integrate these two engines, by WebIDL Binding Generator. This can avoid issues such as Type Checking, Type Converting, and Manage Isolate & Context in the Binding process and increase productivity. You can find out about using Chromium's WebIDL through the link below.
https://www.chromium.org/blink/webidl

What's the benefits of WebIDL Binding Generator?

It has the following advantages.

Code complexity is reduced

The binding code and the implementation of native code are separated so we can keep the code simple. Here's the example of comparing the code complexity when using WebIDL Binding Generator.

It saves a lot of developers efforts.

As mentioned earlier, we don't have to write N-API binding code so it means developers don't need to check issues such as Type checking, type converting and etc. For example, Javascript doesn't support the method overloading so developers have to check the types of arguments, the number of arguments and etc when using N-API.

We showed Example: add() function in the Introductions section. Let's say there are new two Add methods; one has 3 arguments and the other one has one Object type argument. It can be implemented as following(You don't have to understand the following code fully).

#include <napi.h>

Napi::Value Add(const Napi::CallbackInfo& info) {
  Napi::Env env = info.Env();

  if (info.Length() == 1) {
    if(!info[0].IsObject()){
      Napi::TypeError::New(env, "Wrong number of arguments").ThrowAsJavaScriptException();
      return env.Null();
    }
    Napi::Value value1 = info[0].As<Napi::Object>().Get("value1");
    Napi::Value value2 = info[0].As<Napi::Object>().Get("value2");

    if (!value1.IsNumber() || !value2.IsNumber()) {
      Napi::TypeError::New(env, "Wrong arguments").ThrowAsJavaScriptException();
      return env.Null();
    }
    double arg0 = value1.As<Napi::Number>().DoubleValue();
    double arg1 = value2.As<Napi::Number>().DoubleValue();
    Napi::Number num = Napi::Number::New(env, arg0 + arg1);
    return num;
  } else if (info.Length() == 2) {
    if (!info[0].IsNumber() || !info[1].IsNumber()) {
      Napi::TypeError::New(env, "Wrong arguments").ThrowAsJavaScriptException();
      return env.Null();
    }
    double arg0 = info[0].As<Napi::Number>().DoubleValue();
    double arg1 = info[1].As<Napi::Number>().DoubleValue();
    Napi::Number num = Napi::Number::New(env, arg0 + arg1);
    return num;
  } else if(info.Length() == 3) {
    if (!info[0].IsNumber() || !info[1].IsNumber() || !info[2].IsNumber()) {
      Napi::TypeError::New(env, "Wrong arguments").ThrowAsJavaScriptException();
      return env.Null();
    }
    double arg0 = info[0].As<Napi::Number>().DoubleValue();
    double arg1 = info[1].As<Napi::Number>().DoubleValue();
    double arg2 = info[2].As<Napi::Number>().DoubleValue();
    Napi::Number num = Napi::Number::New(env, arg0 + arg1 + arg2);
    return num;
  } else {
    Napi::TypeError::New(env, "Wrong number of arguments").ThrowAsJavaScriptException();
    return env.Null();
  }
}

This code could be optimized but the point is that in order to support Method overloading, code would be complicated and developers should consider argument checking, type checking and etc.

By using the WebIDL binding generator, the code will be implemented as follows.

// Add.nidl
interface Calculator {
    double add(double a, double b, optional double c = 0);
    double add(TwoNumbers n);
}

dictionary TwoNumbers {
    double value1;
    double value2;
}

// Add.cc
double Add(double a, double b, double c = 0) {
    return a + b + c;
}

double Add(TwoNumbers a) {
    return a.value1() + a.value2();
}

WebIDL is easy

Developers no longer need to write N-API binding code so they don't have to learn N-API.
※ IDL is very straightforward and easy to learn. It's like defining C++ headers.

POC(Proof of Concept)

How you can test WebIDL Binding Generator?

You can test it here.

How it works internally?

Our WebIDL Binding Generator works as follows:

• Calling generator() in JS side from binding.gyp
  • When developers declare Command Expansions into 'sources' field in binding.gyp as follows, WebIDL generator is initiated from node-gyp.
• Finding all .nidl files from $PWD
• Parsing all .nidl files by using WebIDL parser
  • In this step, we should parse all found .nidl files. We are internally using WebIDL parser that W3C provides to. After this step, we can get AST(Abstract Syntax Tree) for .nidl files.
• Generating C++ binding code
  • Generates C++ binding code automatically by using parsed data(called AST). The work is similar to the template engine(e.g. Jinja2).

Implementation plan

• We have already a prototype implementation. I'd like to re-implement them into this project getting quality reviews.
• Implement this feature as an optional feature.

Risks

• No risks; This feature is optional. If developers don't want to use this feature, they can easily turn off the flag in binding.gyp.

[digitalinfinity]

@romandev this is great and along the lines of how I envisioned N-API would evolve. One question- can WebIDL generate other targets too other than N-API. Also, does WebIDL support describing cases where the code is not available (for example, if you have an existing shared library, is WebIDL expressive enough to state that the implementation of the function is in that library?)

But I agree, this makes sense to have as an affordance for N-API developers since it simplifies the amount of boiler plate needed considerably. @nodejs/addon-api any thoughts on whether this should be a part of node-addon-api or a separate project?

[mhdawson]

I'll add my 2 cents that I think this looks interesting as well. More tooling on top of N-API and node-addon-api to simplify native development is a great thing.

One of the key questions will be if it should be part of node-addon-api or a module which depends on it. The right answer might also change over time :)

[romandev]

@romandev this is great and along the lines of how I envisioned N-API would evolve. One question- can WebIDL generate other targets too other than N-API. Also, does WebIDL support describing cases where the code is not available (for example, if you have an existing shared library, is WebIDL expressive enough to state that the implementation of the function is in that library?)

Thank you for comment.
I think we can cover the shared library case using extended attribute.

For example,

interface Hello {
    [Implementation="hello_world.so"] void world();
};

I'm not sure how we can implement it right now but we can consider it enough.

But I agree, this makes sense to have as an affordance for N-API developers since it simplifies the amount of boiler plate needed considerably. @nodejs/addon-api any thoughts on whether this should be a part of node-addon-api or a separate project?

In my humble opinion, this WebIDL generator feature should be implemented in node-addon-api repo for the following reasons:

• Make maintenance difficult

  • This feature is tied with node-addon-api. So, when any API face is changed in node-addon-api, the WebIDL generator should also be changed for adapting the API face change. But if there is a breaking change in upstream(node-addon-api), it's not easy to follow the change at the same time in downstream(WebIDL generator).

• Makes users add a new repo or package

  • Adding a new repo or package might not be trivial in project perspective. So, it's not easy for them to overcome entry barriers to use the feature.

• Makes project promotion difficult

  • Let's imagine that there are two repos node-addon-api and webidl-node-addon-api. Their goals are the same. Then, which project should we promote to our users?

• A similar case: Emscripten project

  • Emscripten is also provides WebIDL generator.
  • The WebIDL generator is placed in their repo together.

[creationix]

Has any progress on this been made? I'm interested in using IDL to generate generic V8 bindings for non-node projects. For my particular use case I won't have NAPI or NAN since they are tightly coupled to node, but node support of IDL is interesting nonetheless.

[mhdawson]

@romandev I'm going to close this out since there has not been progress for quite some time. Please let us know if that is not the right thing to do.