-
-
Notifications
You must be signed in to change notification settings - Fork 193
Introduction to Mixins Understanding Mixin Architecture
Before you start writing mixins, it is important to develop an understanding of the basic concepts that allow them to work. This section gives a brief introduction to these concepts. Even though you may be familiar with all of the information presented here I recommend at least skim reading the first 3 sections as they introduce the example case I will be using to demonstrate how mixins are applied, and some peculiar corners of Java and the JVM which are leveraged heavily in mixins.
This is not a tutorial! This introduction is not intended as a tutorial, for more details about mixin implementation consult the mixin example code in the examples in the Sponge repository.
NOTE
If you already have a comprehensive understanding of bytecode, name binding and frankly if you already know your INVOKESPECIAL from your INVOKEVIRTUAL, then feel free to skip to section 4 where mixins themselves are introduced.
In order to be able to think about how mixins work, I will present a canned example. Note that this example is purely made up for the purposes of demonstration and isn't anything like the namesakes in the real code base!
In our canned example, we will be looking at a class EntityPlayer
, whose immediate (and only) superclass is Entity
. We can represent this in a UML-esque manner like this:
Figure 1 - a simple (imaginary) class hierarchy
In mixin jargon EntityPlayer
is the target class, it is the class which the mixin will be applied to.
To flesh out the example, let's add some imaginary fields and methods to our imaginary example classes:
Figure 2 - adding some imaginary fields and methods to our example
This representation is chosen deliberately to represent which members represent our class's public surface area, with public methods and fields jutting outside of the class body since they are visible to other objects. This "view" that the outside world has of our class is an important concept to keep in mind when working with mixins.
Notice that the inherited public methods from Entity
also represent part of our class's publicly visible surface area, and the "ghost" methods getHealth
and setHealth
which are inherited from the parent class represent this presence in the class's overall external appearance.
Before working with mixins, it is vital to have a deep understanding of the two Java keywords this
and super
. That may seem like an odd statement since anyone who has worked with Java for more than five minutes will recognise these keywords and their usage, yet appreciating the subtle implications of both is important if you don't want to go insane when writing mixins.
First let's look at some of the possible invocations and accesses in our imaginary class:
Figure 3 - some possible field and method accesses
There's nothing particularly controversial about this scenario, this.level
, this.update()
and this.food
all seem pretty standard. On the other hand the calls to super.health
and this.health
, and the call to super.onUpdate()
from update
are designed to highlight an aspect of the JVM which isn't obvious when writing Java code.
Ask yourself the following questions:
-
What are the practical implications of qualifying the call to
onUpdate
withsuper
rather than qualifying it withthis
? -
What are the practical implications of qualifying the access of
health
withsuper
rather than qualifying it withthis
?
After all, both qualifications will work exactly the same in practice, right?
The answer to the two questions is:
-
super.onUpdate()
will always call the method inEntity
even if a subclass overrides it, whereasthis.onUpdate()
will, in subclasses which override the method, call the overridden method where appropriate. -
There is no difference, the field in
Entity
will always be accessed from the methodtakeDamage
, even if a subclass "hides" the field by declaring it again.
The underlying reason for this behaviour is that invocations qualified with super
, and all field accesses are statically bound at compile time, this means they always reference the member. Conversely, accesses qualified with this
are dynamically bound, this means they don't resolve their target until they are actually called, allowing subclasses to override methods and have them called when appropriate.
NOTES
As well as invocations qualified with
super
, access toprivate
andstatic
methods are always statically bound as well.In bytecode, statically-bound invocations are represented by the INVOKESPECIAL and INVOKESTATIC opcodes, dynamic calls are represented by the INVOKEVIRTUAL opcode.
Realising the precise nature of these keywords is useful when developing mixins and is the reason for some of the restrictions imposed on mixin classes, more on this later.
I avoided using the word Interface above to describe the publicly visible members in order to avoid confusion with actual Java interfaces, since interfaces themselves play a key role in how mixins can be employed.
To see how interfaces affect our interaction with a class, let's look at what happens if we create an interface which contains a few of the methods in our example, and access those methods via the interface.
Side note: yes this goes completely off the UML rails but UML is not really useful for representing the concepts here, the bottom of this block diagram is the 'visible surface' of a class from the point of view of any other object, the interface is - in effect - sitting "in front of" the public façade of the class and presenting a subset of it.
Figure 4 - a diagram to annoy UML enthusiasts
There are some useful things to make a note of here:
-
Firstly, it's crucial to note that the
getHealth
andsetHealth
in theEntity
class are actually implementing the interface methods, even though the classEntity
has no knowledge of the interfaceLivingThing
, the implication being that there is nothing special about interface methods in a class: as long as the method signatures1 match those in the interface, then the class method is deemed to implement the interface method. It should be clear from this that interface method calls are dynamically bound. -
We also made no changes to either class except for declaring that it
implements LivingThing
. In fact, if Java didn't require us to include theimplements
clause then this program structure would be allowed with no changes to the program at all. What this tells us is that if we can somehow sneakily insert theimplements
clause onto a target class, then provided the methods in our interface exist we will be able to invoke them on the target class.
1 A method's signature is its set of parameters and its return type. For example for the method:
public ThingType getThingAtLocation(double scale, int x, int y, int z, boolean squash) {the signature would be:
(double,int,int,int,boolean)com.mypackage.ThingType
note that we put the parameters in parentheses and the return type on the end. In practice to save space, a more compact syntax is used and in bytecode the above signature would look like this:
(DIIIZ)Lcom/mypackage/ThingType;
You will need to become familiar with bytecode descriptors if you plan to work with Injectors.
The final piece to the puzzle which we are slowly assembling is a useful Java language feature relating to interfaces, namely the fact that you can cast any object reference to any interface and the compiler will happily compile it.
For example, let's say we invent a new interface for objects which can level up, and call it Leveller
, like this:
Figure 5 - what a beautiful day, hey, hey
The following code will happily compile:
public void method() {
// Make a new EntityPlayer
EntityPlayer player = new EntityPlayer();
// This will compile, even though EntityPlayer doesn't
// actually implement the interface, but it will throw
// a ClassCastException at runtime
Leveller lev = (Leveller)player;
// We will never reach this code, but again it will
// compile just fine.
int level = lev.getLevel();
}
We know from the last section that the method getLevel()
in EntityPlayer
can happily implement the interface with no changes, but the fact that the implements
clause doesn't explicitly declare the interface causes the cast to fail at runtime. If we can somehow apply the implements
clause at runtime, then we finally have a viable way of implementing duck typing in Java using interfaces.
"implementing what?"
- you, probably
Duck typing is a method of implicit typing used in dynamically typed languages which allows object members to be accessed or invoked based simply on whether they exist or not. It takes its name from the "duck test" expressed as
When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck.
In other words, if all we care about is that an object has methods quack()
and walk()
then as far as we are concerned it's a Duck
and don't actually care if it's just a very smart Pigeon
, as long as it has the methods then it's a Duck
to us.
If it's still not obvious what's happening here then I suggest reading the Wikipedia article as it covers the concept in detail which is beyond the scope of this introduction.
So what do we know so far?
-
We know that the relationship between classes and interfaces is quite flimsy, and with a little spit and sellotape can be manipulated in a number of ways which are beneficial to us.
-
We know that we can leverage dynamic binding in Java to write code which compiles (although it won't run, yet) and that somehow patching the
implements
clause onto the target object is the key to making this work. -
We know that superclass invocations using the
super
keyword are statically bound at compile time, and this means we have to give extra thought to exactly what we're referring to when we specifysuper
.
One final thing to consider is what happens when a class doesn't implement an interface. Let's add another method to our Leveller
example interface called setLevel()
:
Figure 6 - adding setLevel()
Adding the second method to the interface adds scope for another - different - runtime error, in this case an AbstractMethodError
public void method() {
EntityPlayer player = new EntityPlayer();
// Assume that we can runtime-patch the interface
// declaration onto the EntityPlayer class, allowing
// this assignment to succeed
Leveller lev = (Leveller)player;
// This statement will throw an AbstractMethodError at
// runtime because setLevel(I) is not defined in the
// EntityPlayer class or any of its superclasses.
lev.setLevel(10);
}
Understanding these aspects of Java and the JVM, let's take a look at mixins themselves.
So now we know the basic tasks that mixins must achieve in order to allow us to get other objects to quack:
- Let us apply an interface of our choosing to the target class at runtime
- Let us insert a method implementation for any methods which are declared in the interface but are not present in the target class
First let's look at how we declare a mixin class with EntityPlayer
as its target class:
@Mixin(EntityPlayer.class)
public abstract class MixinEntityPlayer
extends Entity {
}
Yes it really is that simple. Using the @Mixin
annotation defines this class as a mixin and specifies the target class we want to apply it to. Also note that:
-
The mixin class is marked with the
abstract
modifier. Whilst this is not a requirement, it helps when using mixins within an IDE because it means the end user cannot write code which tries to instantiate a mixin class, which would lead to an error at runtime. It also removes the requirement (imposed by the Java compiler) to implement every method within any declared interfaces, which is one of the main goals of mixins. -
The mixin class extends
Entity
, which is the same superclass as our target class. This is important in order to preserve the semantics of any static bindings which are compiled into our mixin class. More details on this later.
If we were to include this mixin in our runtime right now and run the game, the mixin would be applied and absolutely nothing would be changed, this is because we haven't actually declared anything in our mixin. Let's take a look at how we can achieve objective 1 above, and use our mixin to monkey-patch a new interface onto the target class:
@Mixin(EntityPlayer.class)
public abstract class MixinEntityPlayer
extends Entity
implements LivingThing {
}
That's it! Any interfaces declared on the mixin are applied to the target class when the mixin is processed. Let's take a look at the current class hierarchy:
Figure 7 - mixin hierarchy (before application)
While this diagram represents the actual hierarchy of classes we will create, it is actually more useful (and in some more complex cases, vital) to realise that a mixin is not really a class. At runtime, the mixin will be applied to the target class and so it is much more conducive to good thought processes to think of mixins as existing within the target class instead.
After the mixin is applied, the new class hierarchy looks like this:
Figure 8 - class hierarchy (after application)
As we can see, the target class now implements the LivingThing
interface, which now allows our duck typing to work as we wanted:
public void method() {
EntityPlayer player = new EntityPlayer();
// With the mixin applied, this cast succeeds
LivingThing living = (LivingThing)player;
// And because the cast succeeded, we can pass our object
// to other things that require a reference to LivingThing
// such as the method below
if (this.isAlive(living)) {
// hooray
}
}
public boolean isAlive(LivingThing living) {
// We can call getHealth() just fine, because the method
// exists and is accessible via the LivingThing interface
int health = living.getHealth();
return health > 0;
}
Since we've taken care of the first objective and can now successfully apply new interfaces to the target class, let's take a look at the second objective:
- Let us insert a method implementation for any methods which are declared in the interface but are not present in the target class
We'll begin by having our mixin class implement the Leveller
interface, which declares a method not currently implemented in our target class or any of its superclasses:
@Mixin(EntityPlayer.class)
public abstract class MixinEntityPlayer
extends Entity
implements LivingThing, Leveller {
}
producing the following class hierarchy:
Figure 9 - mixin hierarchy (before application)
Because our mixin class is abstract
, this code will happily compile, however any runtime call to the setLevel()
method will result in an AbstractMethodError
as described above. We can fix this by defining the setLevel()
method in the mixin itself:
@Mixin(EntityPlayer.class)
public abstract class MixinEntityPlayer
extends Entity
implements LivingThing, Leveller {
@Override
public void setLevel(int newLevel) {
// TODO implement this method
}
}
Figure 10 - adding a method to the mixin
Now when the mixin is applied the new method will also be patched into the target class:
Figure 11 - class hierarchy (after application)
Our patched target class now fully implements all of the declared interfaces and we can see how easy it can be to add a new method to our target class. At the moment our new method doesn't actually do anything, we'll see how we can remedy this in the next section.
So we now have a way to inject new methods into our target class, but we will fairly quickly encounter a problem with implementing the body of our freshly-injected method: In an ideal world we'd like our new setLevel()
implementation to be able to access the level
variable in EntityPlayer
, but there's a problem... it can't.
Figure 12 - impossible access
We can't access a member of the target class because until the mixin is actually applied, the field doesn't exist! Because the superclass of the mixin is Entity
, it doesn't even help if the field is protected
: as far as the Java compiler is concerned, the field is nowhere to be seen.
However we know that when the mixin is applied that the field will be there, what we need is some way of telling Java "hey, this field will exist, let me access it". Fortunately mixins provide a mechanism for doing exactly this, via the @Shadow
annotation:
@Mixin(EntityPlayer.class)
public abstract class MixinEntityPlayer
extends Entity
implements LivingThing, Leveller {
@Shadow
private int level;
@Override
public void setLevel(int newLevel) {
// Refers to the shadow field above, but will refer
// to the REAL field when the mixin is applied
this.level = newLevel;
}
}
The @Shadow
annotation creates a "virtual field" in the mixin which mirrors its target class counterpart:
Figure 13 - me and my shadow
It is also possible to apply @Shadow
to methods as well, in order to invoke methods which are only defined in the target class, for example say we wanted to call the update()
method immediately after setting the level, we can easily shadow the method and then invoke it from our new setLevel()
method body:
@Mixin(EntityPlayer.class)
public abstract class MixinEntityPlayer
extends Entity
implements LivingThing, Leveller {
@Shadow
private int level;
@Shadow
private void update() {}
@Override
public void setLevel(int newLevel) {
// Set the level value
this.level = newLevel;
// Invoke the shadowed method to update the object state
this.update();
}
}
We would normally declare the shadow method as abstract
simply to avoid having to write a method body, however because it is not possible to declare something as both private
and abstract
for obvious reasons, we simply declare the shadow method with an empty body.
Figure 14 - shadow all the things
The final stop on our tour of the basic features of mixins is a brief look at how superclass accesses are handled within mixins. To begin with, we need to first understand why a mixin class is declared with the same superclass as the target class.
First let's take a quick look at our current class hierarchy:
Figure 15 - state of play
Remember from section 1 that invocations qualified with the super
keyword are statically bound. In the context of our mixin class, if we call super.onUpdate()
as shown in figure 15 then the generated bytecode will reference the onUpdate
method in the Entity
class specifically.
When the mixin has the same parent class as the target class, this is exactly what we want. However it is actually possible to have a mixin inherit from any class in the target class's hierarchy, up to and including Object
.
Let's assume for a minute that EntityPlayer
does not inherit directly from Entity
, but instead an intermediate class EntityMoving
, the mixin class will still extend directly from Entity
:
Figure 16 - expanded hierarchy - note: this diagram is deliberately wrong!
Looking at this new hierarchy, it's now obvious why super.onUpdate()
will appear to be calling the method in Entity
from within the mixin class, but this is where it's important that you ignore what your IDE (and common sense probably) is telling you, and remember that a mixin's point of view is ALWAYS that of the target class!
The problem here is that the intermediate class EntityMoving
has overridden the onUpdate
moving and the functional contract of the class is such that calling onUpdate
in the superclass will actually lead to inconsistent behaviour. When we invoke super.onUpdate()
from the mixin, it must have the same semantics as if the same Java statement were invoked from the target class, and this is indeed the case.
-
In order to preserve the semantic consistency of Java code you type into a mixin, the mixin transformer updates all static bindings in the mixin class as it is applied. This means that in the above example, the call to
super.onUpdate()
correctly invokes the method inEntityMoving
-
This doesn't affect the semantics of the
this
keyword. Which forprotected
andpublic
methods will always use dynamic binding and thus will always invoke the appropriate subclass method.
To get technical, the transformer will process all INVOKESPECIAL opcodes in the mixin and analyse the superclass hierarchy of the target class to find the most specialised version of that method. This process is expensive and is only carried out on "detached" mixins (those mixins whose superclass differs from the target class's superclass). To avoid this processing step, it is recommended that mixin classes have the same superclass as their target wherever possible.
Figure 17 - final hierarchy (mixin applied)
As you can see, after the mixin is applied to the target class, the semantics of the super.onUpdate()
call are updated to be consistent with the target class and all is once again well.
While this introduction covers the basics of mixins, there are many more aspects of mixin functionality to explore, especially when working in an environment where the target classes will be obfuscated before being used in a production environment.