Implement the MVC pattern (data bindings, see #25)

[mikke89]

It's time to show what I have been working on over the last few months, and open it up for discussion.

The model-view-controller (MVC) approach aims to ease the translation between the application data, the displayed document, and user inputs. This is done by separating the responsibilites for handling each of these tasks. See also the discussion in #25 which led me to start this work.

I've written down some documentation which can be found here: RmlUi data bindings documentation.

The documentation includes examples and details, both for the RML and C++ side.

Example

This example shows how to declare the data bindings in RML. Please see the documentation above for more details including how to interact with it on the C++ side.

<p>
	Incoming invaders:
	<input type="range" name="rating" min="0" max="20" step="5" data-value="incoming_invaders_rate"/>
	{{ incoming_invaders_rate }} / min.
</p>
<button data-event-click="launch_weapons">Launch weapons!</button>
<div data-for="invader : invaders">
	<h1 data-class-red="invader.danger_rating > 70">{{invader.name}}</h1>
	<p>Invader {{it_index + 1}} of {{ invaders.size }}.</p>
	<img data-attr-sprite="invader.sprite" data-style-image-color="invader.color"/>
	<p>
		Shots fired (damage): <span data-for="invader.damage"> {{it}} </span>
	</p>
</div>
<h1 data-if="invaders.size == 0">It's all safe and sound, sir!</h1>

Notice the the small expressions used in some places? We can handle these small little expression inside RmlUi now, which greatly increases the usability of this feature.

One breaking change is that {{ and }} can no longer be used inside RML documents for other purposes than data bindings.

Discussion

Just to be clear, it is not ready to be merged just yet. There is at least some polishing needed, and I want to get some more experience with using it myself.

How do you like the included features, the syntax, the general workflow? All feedback is very much welcome :)

[viciious]

Excuse my ignorance but I can't really comment on this this since I'm not a guru on modern web techniques and I don't really understand what's going on here :) From what I understand this thing allows you to bind certain properties or attributes to C++ variables. If that's the case, then I guess this feature is not for me since I'm not coding UI's in pure C++: I've got a huge AngelScript binding library for this purpose. Or do scripted languages benefit from this feature too?

[barotto]

Impressive, very nice. As a first version it seems quite good already. I'm sure this feature will be very useful in many cases.

Speaking of syntax, the curly braces remind me of Smarty (which I use regurarly), although Smarty uses single braces with no spaces in between, like so {term}.
Other similarities with Smarty's syntax are in the data variables and data expressions, which are quite close to RmlUI.

A strong difference though is in the data views, where Smarty works using {...}html{/...} blocks.
If I were to translate your example to a Smarty template I would do something like this (not tested):

<p>
	Incoming invaders:
	<input type="range" name="rating" min="0" max="20" step="5" value="{$incoming_invaders_rate}"/>
	{$incoming_invaders_rate} / min.
</p>
<button onclick="launch_weapons()">Launch weapons!</button>
{foreach $invaders as $invader}
<div>
	<h1 {if $invader.danger_rating > 70}class="red"{/if}>{$invader.name}</h1>
	<p>Invader {$invader@iteration} of {$invaders|@count}.</p>
	<img sprite="{$invader.sprite}" style="image-color={$invader.color}" />
	<p>
		Shots fired (damage): 
		{foreach $invader.damage as $damage}
		<span> {$damage} </span>
		{/foreach}
	</p>
</div>
{/foreach}
{if $invaders|@count == 0}
<h1>It's all safe and sound, sir!</h1>
{/if}

Question: how can I express if...elseif...else in RmlUi? Reading the documentation I suspect you can't, at least not easily.
Maybe Smarty can be of some inspiration?

The data binding mechanism on the C++ side seems to follow Smarty's principles on the PHP side, although in PHP it's much more convenient as you don't need to register anything, you just assign what you need to a Smarty variable.
For example (hypotetical C++ version of Smarty):

constructor.assign("invaders", &invaders_data.invaders);

and then you would have access to the array and the Invader struct inside the rml template, that's it. But I suspect this would be a bit tricky to do in C++ (although I'm definitely not a C++ template metaprogramming expert, so it might be dead simple to Bjarne Stroustrup or Herb Sutter).

[mikke89]

Excuse my ignorance but I can't really comment on this this since I'm not a guru on modern web techniques and I don't really understand what's going on here :) From what I understand this thing allows you to bind certain properties or attributes to C++ variables. If that's the case, then I guess this feature is not for me since I'm not coding UI's in pure C++: I've got a huge AngelScript binding library for this purpose. Or do scripted languages benefit from this feature too?

It should be fully possible to create an API inside scripting languages as well. See eg. this for some inspiration. We may need some additional API on the C++ side, so I'd be interested to hear how it goes if you decide to try this out. :)

@barotto Thanks! I haven't heard about Smarty, but I did consider a syntax similar to what you propose. However, I found that attaching views to particular elements make things a lot simpler, allowing us to separate the data binding behavior from the normal behavior of RmlUi more easily. Also, attaching the views to single elements establishes a clear anchor point in the document.

Previously I didn't think the else and else-if were needed, because you can create equivalent logic using data expression in an if-view. Now that you bring it up, I can see the convenience of it, so I'll go ahead and add them as well :)

With regards to registering types. Unfortunately, this is necessary in C++, at least until we get static reflection. Maybe in 10 years ;) At least this is true for structs. For arrays we could go insane with templates and SFINAE, or use concepts (but that's C++20). I'd rather we be explicit about it than do fancy things like that. I tried making it as easy as possible though, and it is just a one-liner:

data_model_constructor.RegisterArray<std::vector<int>>();

Call that and you are done.

[uniquejack]

I've already tried this branch and it is AWESOME! It makes the developing of the UI much easier!

Technically talking we should be able to create controllers (registering a model with all its properties) also in our scripting languages. CoherentGT, for example, permits the creation of a model with both methods (C++ and JS). At that point we could implement a UI with a MVC design pattern through scripting languages just like BeamNG do: they are using ChromiumEmbedded + AngularJS but the idea behind it is the same.

[mikke89]

Thanks for the nice feedback!

I agree creating models and variables in scripts would be nice. I think it should be possible to implement this already with the current API. For a Lua implementation, I will leave this to somebody more experienced with Lua. I'd be interested to hear how it goes though, please let me know if there is anything needed on the library side.

[mikke89]

Regarding else-if and else, actually this turned out to be a bit complicated due to the dependency on other elements. I have something working for it locally, but it increases the code complexity quite substantially, so I'd rather we stay with just if and use data expressions to recreate the logic, at least for now.

[viciious]

Regarding else-if and else, actually this turned out to be a bit complicated due to the dependency on other elements. I have something working for it locally, but it increases the code complexity quite substantially, so I'd rather we stay with just if and use data expressions to recreate the logic, at least for now.

How about if-not? :P

[viciious]

Thanks for the nice feedback!

I agree creating models and variables in scripts would be nice. I think it should be possible to implement this already with the current API. For a Lua implementation, I will leave this to somebody more experienced with Lua. I'd be interested to hear how it goes though, please let me know if there is anything needed on the library side.

I could try to come up with something for AngelScript but I've got no idea where to start :)

[mikke89]

How about if-not? :P

Uhm, yes, but ! is fewer characters :P

I could try to come up with something for AngelScript but I've got no idea where to start :)

That would be very cool.

First you need some way of creating a model from scripts. So eg. (pseudo code)

<script>
bindings = {
  title: 'Hello world',
  quantity: 50,
  max: 200
}
context.createModel('my_model', bindings)
</script>

This script should run before the document body is loaded. In your script bindings on the C++ side, this should create the model and do the bindings on the provided name-value object. Generally, you can follow the examples in the documentation for how to set up the model. Something like the following perhaps (pseudo-ish code):

struct ModelData {
	UnorderedMap<String, Variant> data;
	void Get(const String& name, Variant& out_value) {
		out_value = data[name];
	}
	void Set(const String& name, Variant& value) {
		data[name]= value;
	}
} model_data;

String model_name = get_first_argument();

if (auto model_constructor = context->CreateDataModel( model_name ))
{
	// Depends on how you retrieve / iterate over objects in your script language.
	for ( auto& name_value_pair : get_second_argument() )
	{
		String name = name_value_pair.first;
		Variant value = name_value_pair.second;
		model_data.Set(name, value);
		
		constructor.BindFunc(
			name,
			[name](Variant& out_value) { model_data.Get(name, out_value); }, // Getter
			[name](const Variant& value) { model_data.Set(name, value); } // Setter
		);
	}
	model_handle = model_constructor.GetModelHandle();
}
// Store the handle and data for later.

I mean, this all depends on exactly how you interface with the scripting language and how you want to store the data. Perhaps a better solution is to store the data as variables inside the scripting languages which can then also be used in script functions. If you can get stable pointers to variables inside the script then they can be used without getters and setters. I don't know angelscript so this you will have to figure out :)

Finally, during context.Update() (before the document updates) you will have to call

model_handle.Update();

That should hopefully be enough to get some simple examples up and running.

And then you may want to add support for arrays and structs. Hmm, for structs we may need a slightly different API to be able to dynamically add members... And there is a question of how to remove bindings / models upon closing the document, which is not possible right now :P

Let me know how it goes :)

[uniquejack]

How about if-not? :P

Uhm, yes, but ! is fewer characters :P

I could try to come up with something for AngelScript but I've got no idea where to start :)

That would be very cool.

First you need some way of creating a model from scripts. So eg. (pseudo code)

<script>
bindings = {
  title: 'Hello world',
  quantity: 50,
  max: 200
}
context.createModel('my_model', bindings)
</script>

This script should run before the document body is loaded. In your script bindings on the C++ side, this should create the model and do the bindings on the provided name-value object. Generally, you can follow the examples in the documentation for how to set up the model. Something like the following perhaps (pseudo-ish code):

struct ModelData {
	UnorderedMap<String, Variant> data;
	void Get(const String& name, Variant& out_value) {
		out_value = data[name];
	}
	void Set(const String& name, Variant& value) {
		data[name]= value;
	}
} model_data;

String model_name = get_first_argument();

if (auto model_constructor = context->CreateDataModel( model_name ))
{
	// Depends on how you retrieve / iterate over objects in your script language.
	for ( auto& name_value_pair : get_second_argument() )
	{
		String name = name_value_pair.first;
		Variant value = name_value_pair.second;
		model_data.Set(name, value);
		
		constructor.BindFunc(
			name,
			[name](Variant& out_value) { model_data.Get(name, out_value); }, // Getter
			[name](const Variant& value) { model_data.Set(name, value); } // Setter
		);
	}
	model_handle = model_constructor.GetModelHandle();
}
// Store the handle and data for later.

I mean, this all depends on exactly how you interface with the scripting language and how you want to store the data. Perhaps a better solution is to store the data as variables inside the scripting languages which can then also be used in script functions. If you can get stable pointers to variables inside the script then they can be used without getters and setters. I don't know angelscript so this you will have to figure out :)

Finally, during context.Update() (before the document updates) you will have to call

model_handle.Update();

That should hopefully be enough to get some simple examples up and running.

And then you may want to add support for arrays and structs. Hmm, for structs we may need a slightly different API to be able to dynamically add members... And there is a question of how to remove bindings / models upon closing the document, which is not possible right now :P

Let me know how it goes :)

Thanks for the example, I'm going to implement something like that with Javascript.
Btw when can we expect a merge with the master branch?

EDIT: I think I'm about to make it work. I was wondering if it could be possible to "extend" an already created model, all I need is a method to get the model constructor by name so I can add some getters and setters to it.

[mikke89]

Thanks for the example, I'm going to implement something like that with Javascript.
Btw when can we expect a merge with the master branch?

Great! I guess we can merge it soon, I wouldn't consider it API stable yet but we could also keep working on it from the master branch. I'll give it some more time at least :)

EDIT: I think I'm about to make it work. I was wondering if it could be possible to "extend" an already created model, all I need is a method to get the model constructor by name so I can add some getters and setters to it.

Yeah, I was thinking the same thing, it's on my todo-list :)

[viciious]

<script> bindings = { title: 'Hello world', quantity: 50, max: 200 } context.createModel('my_model', bindings) </script>

Hm, I see. In case of AngelScript though this particular implementation is problematic because handles (pointers) to primitive types are not allowed. There's a dictionary type that stores key-value pairs, where the key is a string, and the value can be of any type, including handles and primitive types. But it comes with additional overhead. I'll need to give this additional thought.

[mikke89]

Hm, I see. In case of AngelScript though this particular implementation is problematic because handles (pointers) to primitive types are not allowed. There's a dictionary type that stores key-value pairs, where the key is a string, and the value can be of any type, including handles and primitive types. But it comes with additional overhead. I'll need to give this additional thought.

I see. It might help to use getters and setters. Let me know if there is anything that could be improved on the library API side.

By the way, I've added the ability to retrieve an existing DataModel (constructor) from the context, in addition to removing the model.