Generalized Script extensibility?

[JeroMiya]

I've recently worked on a proof of concept integration of X_ITE within a Blazor application. I've been able to successfully wrap a small subset of the JS scene access interface so that C# wasm code can manipulate the X3D scene via C# code running in the browser. In other words, implementing the C# scripting functionality by treating the X_ITE browser as an "external" browser, as the documentation calls it. This works fine and is promising.

It got me thinking, however. The X3D Script node is "theoretically" language agnostic. But most X3D browser implementations (including X_ITE) only support ECMASCRIPT, with some older ones supporting Java (I think Xj3D?). For example, this would be a Script node that references a java class:

    <Script DEF="ColorChangerScript" url="JavaColorChanger.java">
      <field name="color_changed" accessType="outputOnly" type="SFColor"/>
      <field name="touch_event" accessType="inputOnly" type="SFBool"/>
    </Script>

So I was thinking, WebAssembly has allowed several languages (including Java) to be compiled to a form that can run in the browser, and they usually provide a JS-Wasm interop functionality. I'm most interested in Blazor for C# applications, but there are several from rust, python, ruby, and even Java. And even without WebAssembly, there are visual scripting languages like scratch or node-based languages, or various interpreted languages with the interpreter ported to JS (perhaps via wasm?).

So the question in my mind would be this: would it be possible for X_ITE to implement some kind of extensibility interface that allowed various other scripting languages to be implemented in the browser (some perhaps using WebAssembly, maybe others using interpreted or visual scripting languages)? What would that look like?

My rough idea would be that X_ITE would provide an API to register an extension to the scripting system that would allow it to declare that it supported a particular MIME type, or perhaps register a url prefix for the Script url parameter, so for example for a hypothetical rust extension might register the "rust:" schema/prefix, and you might have a Script node like this:

    <Script DEF="ColorChangerScript" url="rust:JavaColorChanger.wasm">
      <field name="color_changed" accessType="outputOnly" type="SFColor"/>
      <field name="touch_event" accessType="inputOnly" type="SFBool"/>
    </Script>

Or the extension could provide a callback that X_ITE calls to ask if it understands a particular url value.

Then, the script extension would be able to interpret the url property in some language specific way. For instance, for a hypothetical java-wasm extension, it might implement the existing Java standard to support existing X3D content that uses Script nodes referencing Java class files.

For a hypothetical dotnet wasm extension, there isn't an existing standard, but maybe it might be the full path to the type:

    <Script DEF="ColorChangerScript" url="dotnet:MyExample.Namespace.JavaColorChanger">
      <field name="color_changed" accessType="outputOnly" type="SFColor"/>
      <field name="touch_event" accessType="inputOnly" type="SFBool"/>
    </Script>

Or perhaps the extension has a runtime where the C# side registers named script node factories, and the URL is just the registered name of the factory. There could be several options.

Then, the extension would read that value, initialize its own "runtime" or interpreter, and facilitate the lifecycle/communication between the X_ITE browser object and the script runtime, and also manage lifecycle events and so on.

Has this been discussed before? Is it feasible?

[create3000]

It is already possible to to add real new node classes or change the behaviour of an existing node class.

const browser = X3D .getBrowser ();

await browser .loadComponents ("Scripting");

const Script = browser .getConcreteNode ("Script"); // Or use X3D.Script which is available now.

class MyScript extends Script { }

browser .updateConcreteNode (MyScript);

// All newly created Script nodes will be now of class MyScript

You should take a look at https://github.com/create3000/x_ite/blob/main/src/x_ite/Components/Scripting/Script.js to see what is needed and how node classes look.

Overloading or recreating a node class is, of course, the most challenging thing there is, so think carefully about whether you want to do that.

Such things are rarely documented because you should know what you do and to know the code base very well.