Enable transform chaining by making scaled/rotated/translated consistent

[bluenote10]

Describe the project you are working on:

A guitar practicing application (here is an old version but in VR).

I suspect the reasons why I'm facing this issue more than others are:

• Almost all geometry/meshes are procedurally generated.
• Because of usage of MultiMeshInstance I can't rely less on node hierarchies, but have to compute complex transformation chains manually.
• I like fluent interfaces ;)

However the issue I'm facing is a fairly fundamental one, and can affect almost any project.

Describe the problem or limitation you are having in your project:

Basically every time I try to chain .scaled/.rotated/.translated I end up with a bug. I had raised this issue before godotengine/godot#34329, which has been closed by adding a line to the documentation. This line didn't help much, I still run into the same problem a lot. I finally had the time to analyze why that is, and since 4.0 is the chance to get things right, here is my proposal.

Note: I'm discussing the topic based on Transform, but both the problem and solution would also apply for Transform2D as far as I can see.

Also feel free to read the much shorter proposal (down below) first, and only dive into the lengthy reasoning behind it (above) if need be ;).

Terminology

Chaining transformation always requires to be aware of left-to-right or right-to-left thinking, because the "mathematical reading order" is typically opposite to the "transformation order". For instance

x' = C · B · A · x

first transforms x by A, then by B, then by C, opposite to how the mathematical equation is typically written. Put another way:

• Transform left multiplication

  M' = A · M
  

  means that A is applied after M is applied.

• Transform right multiplication

  M' = M · A
  

  means that A is applied before M is applied.

Current behavior

Currently the behavior is a mix of left and right multiplication.

• Scaled:

  var M_new = M.scaled(... S ...)
  

  has left multiplication semantics (i.e., happens after in transformation order):

  M_new = S · M
  

• Rotated:

  var M_new = M.rotated(... R ...)
  

  has left multiplication semantics (i.e., happens after in transformation order):

  M_new = R · M
  

• Translated:

  var M_new = M.translated(... T ...)
  

  has right multiplication semantics (i.e., happens before in transformation order):

  M_new = M · T
  

Issue 1: Hard to read

Because of mixing left and right multiplication, I find it fairly hard to look at chained expressions and come up with the underlying mathematical order. Going from the code to the mathematical expression cannot be done by just reading in one direction, but rather requires to switch between left-to-right and right-to-left thinking. For instance:

var M = Transform.IDENTITY\
    .scaled(... S ...)\
    .translated(... T ...)\
    .rotated(... R ...)

is equivalent to (if I didn't get it wrong again)

M = R · S · T

Note how R has moved from last to first, S has moved to the middle, and T ended up at the end. The result feels almost like a random shuffle of the order written in the code. Doing such transformations on longer expressions is a challenging (and unnecessary) mental exercise.

Issue 2: Hard to write

The problem is even more tricky the other way around, when trying to convert a mathematical expression into code.

Example 1: Imagine your goal is to write the following purely with chaining:

M = S · T · R

As far as I can see this actually cannot be written purely with chaining, because having the .translated in the middle breaks the right-to-left flow, and there is no way get the R in the right position.

The only way to write it is in a non-chained way, for instance:

var M = Transform.IDENTITY\
    .scaled(... S ...)
    .translated(... T ...)
M *= Transform.IDENTITY.rotated(... R ...)

Example 2: Imagine implementing longer transform chains like:

M = R_2 · T_2 · R_1 · S_2 · T_1 · S_1

Trying to work out the code becomes more and more awkward, because it is necessary to split the expression into subgroups at each T_*, which break the right-to-left flow. The individual groups can be assembled right-to-left, but need to be assembled
in an outer multiplication left-to-right. An alternative is to manually implement translation left multiplication with the trick to use temporary.offset += T_*. In any case the resulting code is much less clear than a full chaining expression (if .translated would do left-multiplication as well):

var M = Transform.IDENTITY\
    .scaled(... S_1 ...)\
    .translated(... T_1 ...)\
    .scaled(... S_2 ...)\
    .rotated(... R_1 ...)\
    .translated(... T_2 ...)\
    .rotated(... R_2 ...)

Issue 3: Performance aspects

In general writing transforms as chains is faster than using full transform1 * transform2 expressions, because the implementation can exploit the particular matrix properties of .scaled/.rotated/.translated. However, because of the error prone chaining semantics, I have basically replaced many transform chains by transform product expressions, which has performance drawbacks.

I had to refresh my memory about the differences, in case you are interested in the details:

All possible transform operations

Translation

Left multiply

Right multiply

Scale

Left multiply

Right multiply

Rotation

Left multiply

Right multiply

Generic transform

Left multiply (rhs only)

Right multiply (rhs only)

Counting the number of floating point operations gives:

Operation	# Floating point operations	available
Translation (left multiply)	3
Translation (right multiply)	18	*
Scale (left multiply)	12	*
Scale (right multiply)	9
Rotation (left multiply)	60	*
Rotation (right multiply)	45
Full transform multiplication	60	*

It is interesting to see how much more costly a full transform1 * transform2 (60 ops) is compared to a simple translation left multiply (3 ops). The other aspect I vaguely remembered: For scale/rotate the faster operation is right multiplication, whereas for translation it is left multiplication. If performance is critical, it can be helpful to build a transform exactly in the way that minimizes floating point operations. Unfortunately, the interface in Godot is not only inconsistent, but also offers only the less efficient variants.

Describe the feature / enhancement and how it helps to overcome the problem or limitation:

The solution I'm proposing is to make the interface consistent and offer all possible operations:

• .scaled performs left multiplication
• .rotated performs left multiplication
• .translated performs left multiplication
• .pre_scaled performs right multiplication
• .pre_rotated performs right multiplication
• .pre_translated performs right multiplication

This is also the solution chosen by Eigen (possibly the most famous library in that area) just the other way around because of using participles instead of infinitives, and to be less of a breaking change. What "pre" means is a matter of convention anyway, and up to the documentation to communicate.

Thus, in terms of documentation it is key to clearly describe what these functions do mathematically. Currently the docs do not even clearly say whether they are performing left or right multiplication, I only figured it out after reading the C++ sources. I could contribute some of the stuff above to the docs.

In terms of breaking changes this would add one item to the Godot 4.0 migration guide: Replace translated by pre_translated.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

Should be covered by the section above.

If this enhancement will not be used often, can it be worked around with a few lines of script?:

It affects Transform / Transform2D so the enhancement is probably used often.

Is there a reason why this should be core and not an add-on in the asset library?:

Mainly to avoid headaches for others.

[Calinou]

Thus, in terms of documentation it is key to clearly describe what these functions do mathematically. Currently the docs do not even clearly say whether they are performing left or right multiplication, I only figured it out after reading the C++ sources. I could contribute some of the stuff above to the docs.

Please open a pull request for that 🙂

[aaronfranke]

Operation	# Floating point operations	available
Translation (left multiply)	3
Translation (right multiply)	18	*

As discussed here, both of these are available. You can achieve that behavior with t.origin += whatever so there's not much of a reason to have a method for it.

var M = Transform.IDENTITY\
    .scaled(... S_1 ...)\
    .translated(... T_1 ...)\
    .scaled(... S_2 ...)\
    .rotated(... R_1 ...)\
    .translated(... T_2 ...)\
    .rotated(... R_2 ...)

You listed this, and a similar example, under sections called "Hard to read" and "Hard to write". This seems like a pretty weak argument to me, if you are chaining 6 methods after each other then that's not very readable anyway. It would make more sense to calculate each of the transforms that you need, and give them names using variables (ex: for an FPS game, for calculating the camera's transform, one could be called body and one could be head).

The solution I'm proposing is to make the interface consistent and offer all possible operations:

• .scaled performs left multiplication
• .rotated performs left multiplication
• .translated performs left multiplication
• .pre_scaled performs right multiplication
• .pre_rotated performs right multiplication
• .pre_translated performs right multiplication

We only need one translated method, since the other is trivial using t.origin += whatever. The rest make sense. EDIT: Actually, having both is fine, it's consistent to have both versions of each.

[bluenote10]

As discussed here, both of these are available. You can achieve that behavior with t.origin += whatever so there's not much of a reason to have a method for it.

As I tried to explain (An alternative is to manually implement translation left multiplication ...), this is not a chainable, which is what the proposal is about. The table was mainly summarizing available chainable operations.

By not chainable I mean that if you want a chain of say

• scale (left multiply)
• translate (left multiply)
• rotate (left multiply)
  you'd have to "stop" the expression in between the scale and the rotate because you cannot write the left multiply version of translate as an expression, but a statement instead. So instead of 1 chain expression you end up with 3 statements.

This was basically the primary motivation to open the proposal: scale and rotate are already available as left-multiplies, but translate isn't, so you really can't chain the three with same semantics at all -- or rather -- you're lured in to squeeze the available translate into a chain, but you're not getting what you'd expect because it switches the operation associativity.

Other reasons for having all 6:

• Leaving out one of the 6 feels incomplete / inconsistent.
• When a user thinks about building a transform chain, it requires extra mental effort to remember that there is one special case of left multiplying a translation matrix which happens to be equivalent to t.origin += whatever. Unless you have absolutely memorized this fact, a user may first have to take a pen and paper and do the math to see that this particular matrix multiplication really degrades into just the addition. For instance, I personally can never remember whether it is the right or the left multiplication which becomes trivial, and I'd really prefer the library to help me with getting this right.

[andre-caldas]

I think you are making a mistake. Transform2D is about affine transformations, as you already know, since you are using what is called an augmented matrix.

Now, I think that your mistake is interpreting scaled(Vector(a,b)) or rotated(theta) as being equivalent to multiplication on the left by the matrices

[a 0 0]
[0 b 0]
[0 0 1]
and
[cos(theta) -sin(theta) 0]
[sin(theta) cos(theta) 0]
[................................. 1].

But it is not true! If you think about it, scaled() and rotated() are ill defined. There is no "universal rotation" in a plane. What there is, is rotation "about some point"!!! The same is true for scaling. There is no "universal scaling" in the plane. There is scaling "about some point"! When you scale, only one point is kept fixed. Which point are you trying to keep fixed when you calculate t.rotated(theta)? If you do with matrices and multiply those matrices above "by the left", you are rotating about the point (0,0) in your global system of coordinates. When you do t.scaled(Vector2d(2,2)), it is just the same: you are scaling and keeping the global origin fixed. What point do you want to scale about? The origin of the global system of coordinates? Or the origin of the local system?

So, I think rotated() and scaled() mean rotation and scaling about the "origin" of the local system of coordinates. This means that, from the left you should not multiply by the matrices you are multiplying.

This means that for each different value of Transform2D.origin, you have a different matrix you should use to "multiply by the left". For "scaled(Vector2(a,b))", it would be:

[a 0 p]
[0 b q]
[0 0 1]
where p = origin.x * (1 - a), and q = origin.y * (1 - b).

If you want to scale by the global origin, all you have to do is scale the Transform2D.origin as well! And the same goes for rotation. There is no need to have pre_scaled or pre_rotated. This has nothing to do with "left" and "right". This is just because "scaled()" and "rotated()" mean scaled and rotated about the local origin.

[bluenote10]

I think you are making a mistake.

How come all other libraries out there do it that way?

When multiplying matrices there are simply two ways, from the left and from the right. And when it comes to "chaining", i.e., putting multiple of them together in a chain, associativity is crucial. Having all 6 operations available in consistent way is strictly superior in functionality than having only 3 -- even more so when these three mix left and right multiplication semantics.

Look at code like this. The most intuitive interpretation of this code is that the order of these three lines corresponds to the order the matrices are multiplied, which is not the case (so either you likely see a bug here, or you have it look like a bug while being intentional).

Our brains as developers have hard enough tasks to solve, there is no need to make it any more harder with inconsistencies in associativity and non-intuitive order flipping.

[andre-caldas]

Edit: I just saw your youtube video!! That is a very interesting app!!! :-)
It is a shame I do not have VR equipment. :-(

---

This has nothing to do with the order of matrix multiplication. You are multiplying by the wrong matrix. Associativity continues working.

When you scale an object, it should NOT go far away from the origin. It should stay in the same position and scale the object.

Associativity continues working because associativity has to do with "function composition". This continues working, as transformations are being applied when they are invoked. The examples you are using:

1. Rotation about the local origin.
2. Scaling by the same factor in both directions about the local origin.
3. Translation of the local origin in relation to the global origin.
   Those operations ALWAYS commute. Associativity WORKS.

But rotation and scaling about the local origin are represented by different matrices if you ate talking about different origins. This is giving you the impression that multiplication is being done by "the right". It is not being done "by the right". Those operations listed above commute. From the right, the correct matrix is the matrix with (0, 0, 1) in the last column, because from the right, the origin of the coordinate system is the local origin. But from the left, the origin of the coordinate system is the global origin. Since you are not rotating about the global (parent's) origin, you are rotating about the local origin, matrix is different. You are multiplying by the wrong matrix. I found this stackoverflow question about scaling about a particular point. I insist that scaling about the parent's origin is not the expected behaviour.

Any operation done "by the right" can be done "by the left" by a suitable, possibly different, matrix. The problem here is that the matrix you are using only makes sense "from the right", because in this context the point of rotation is the origin of the coordinate system. If you do it from the left, then the point of rotation is no longer the origin of the coordinate system anymore. In this context, the origin of the coordinate system is the parent's origin. Your examples SEEM to show that the order of operations are being messed with, because those operations in fact commute. This is giving you the impression that multiplication is being done "by the right". If you do your experiments with a scaling in only one direction (i.e.: scaled(Vector2(2,1)), you will realize that those transformations are NOT being done "by the right" in the order you think they are. Because, in this case, rotation and scaling do not commute.

The bahaviour you seem to want is not intuitive, as you seem to want that scaling and rotation of an object changes its position relative to its parent's coordinate system. This is not what is supposed to happen. The origin of the local coordinate system is supposed to stay in place, and the changes are supposed to be applied only on the axis (i.e.: variables x and y of Transform2D).

If you want the scaled/rotated object to move away from the parent's origin, you can:

1. Apply the scaling/rotation to both: origin and "linear part" (that is, origin, x and y variables).
2. Change the parent's scale/rotation. (this would affect all siblings, though)

Maybe, if you show a simplified piece of your (real) code and what you are trying to achieve (graphically), it would be easier to talk.

[bluenote10]

Your examples SEEM to show that the order of operations are being messed with, because those operations in fact commute.
Translation of the local origin in relation to the global origin. Those operations ALWAYS commute. Associativity WORKS.

No, if you have a general transformation matrix M than no operation commutes. Even for translations it is different if you do M · T (translate first by T, followed by transforming with M) or T · M (apply M first, followed translating with M).

You are multiplying by the wrong matrix.

There is no right or wrong. It all depends on what you want to achieve.

If you do it from the left, then the point of rotation is no longer the origin of the coordinate system anymore. In this context, the origin of the coordinate system is the parent's origin.

Yes, and there are use valid cases for that.

Maybe, if you show a simplified piece of your (real) code and what you are trying to achieve (graphically), it would be easier to talk.

This is not about my code. It is just about that fact that the current behavior is highly confusing, and in contrast to all other libraries out there. Just try to infer the a mathematical equation from the three lines of the linked example.

---

I don't think this is leading anywhere. All I can say is:

• There are use cases for all 6 variants. In some situations you want to apply an existing M first and then apply a specific translation/rotation/scaling. In others cases, what you want to achieve requires translating/rotating/scaling first, and then applying M. There is no right or wrong here.
• Having 6 operations is better than 3.
• All other libraries I have worked with work that way, and none of them has this inconsistency.

Take the Eigen library as an example, which is a de-facto standard for linear transformation e.g. in robotics, where complex transformation chains arise frequently e.g. when traversing from one robot component/sensor to another (my daytime job). Do you really want to argue that a library that is praised for its design got it all wrong? I can tell you that any programmer with a background in Eigen will be massively confused by the current behavior, and from what I recall from my Java/Scala/GLM/Python days it was the same there.