Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lambda inlining optimization #353

Open
wants to merge 87 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
274fd19
Added lambda inlining
TimothyGeerkensGuardsquare Aug 3, 2023
043be3a
Fixes from inlining lambdas in the klox compiler
MaartenS11 Aug 4, 2023
c098ad8
Added test for retracer + addressed Oberon's comment from index 1 to 9
TimothyGeerkensGuardsquare Aug 4, 2023
7922f11
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
MaartenS11 Aug 7, 2023
bb66672
Fixed regression from merge
MaartenS11 Aug 7, 2023
573995e
Removed some unused fields from the RecursiveInliner
MaartenS11 Aug 7, 2023
e4ce1e0
Removed import .* in LambdaLocator
TimothyGeerkensGuardsquare Aug 7, 2023
70e5966
Simplified the RecursiveInliner a bit
MaartenS11 Aug 7, 2023
aabc5bd
Cleaned up RecursiveInliner by moving some code into a new CalledMeth…
MaartenS11 Aug 7, 2023
f4a22fa
Move some code into IndyLambdaImplVisitor class
MaartenS11 Aug 7, 2023
7b2146e
Remove argument from RecursiveInliner because it isn't really needed …
MaartenS11 Aug 7, 2023
3573c19
Reduce line length a bit by putting index calculation in a variable
MaartenS11 Aug 7, 2023
025571c
Remove wildcard imports from RecursiveInliner
MaartenS11 Aug 7, 2023
845d044
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
MaartenS11 Aug 7, 2023
13b38d5
Removed visitor staircase in LambdaLocator
TimothyGeerkensGuardsquare Aug 7, 2023
fb9fe7d
Removed wildcards in import
TimothyGeerkensGuardsquare Aug 7, 2023
cf74760
Cleaned up the while loop in the LambdaUsageFinder by using the new I…
MaartenS11 Aug 7, 2023
2090514
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
MaartenS11 Aug 7, 2023
fb5eedb
Merge pull request #1 from MaartenS11/klox-fixes
MaartenS11 Aug 7, 2023
6ae5fc3
Removed unused class and method
TimothyGeerkensGuardsquare Aug 7, 2023
e90c5c9
merge
TimothyGeerkensGuardsquare Aug 7, 2023
33aa83d
Add more comments to the RecursiveInliner
MaartenS11 Aug 7, 2023
1d49b49
removed print and added logging
TimothyGeerkensGuardsquare Aug 7, 2023
cb60c13
refactored test and added performance tests
TimothyGeerkensGuardsquare Aug 7, 2023
3d3bf8f
Only stop inlining if we actually have the current lambda in one of t…
MaartenS11 Aug 7, 2023
8c354ab
refactored util
TimothyGeerkensGuardsquare Aug 8, 2023
813be34
Use argument index from LambdaUsageFinder instead of searching in the…
MaartenS11 Aug 8, 2023
f155107
RecursiveInliner should also use the argument index from the lambda u…
MaartenS11 Aug 8, 2023
7d82d2d
Remove findFirstLambdaParameter method since it is no longer needed
MaartenS11 Aug 8, 2023
047719c
WIP commit (please amend)
MaartenS11 Aug 8, 2023
5cc6748
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
MaartenS11 Aug 8, 2023
1f5c85b
Merge pull request #2 from MaartenS11/recursive-inliner-adjustments
MaartenS11 Aug 8, 2023
54233a5
Put stop condition that was duplicated into a method
MaartenS11 Aug 8, 2023
ef37241
Don't hardcode the MethodInliner max resulting code size
MaartenS11 Aug 8, 2023
beeadd5
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
MaartenS11 Aug 8, 2023
343df22
Fixed cast removal so it works in all cases
MaartenS11 Aug 8, 2023
71213fe
Reverted some minor accidental changes
MaartenS11 Aug 8, 2023
42493c6
Cleaned up a bit
MaartenS11 Aug 8, 2023
8cfd505
Merge pull request #3 from MaartenS11/more-fixes
MaartenS11 Aug 8, 2023
cff6565
Conditionally inline lambdas
MaartenS11 Aug 9, 2023
5644a9d
Added logging when a lambda has been inlined and only print method le…
MaartenS11 Aug 9, 2023
48c52e5
Log the consuming method and invoke method so the use knows which met…
MaartenS11 Aug 9, 2023
badf492
Merge pull request #4 from MaartenS11/inline-conditionally
MaartenS11 Aug 9, 2023
61e1e98
Fixed typo in comment
MaartenS11 Aug 9, 2023
00664b2
Removed some unnecessary comments from the tests
MaartenS11 Aug 9, 2023
fca2cde
Fixed typo in comment
MaartenS11 Aug 9, 2023
f8f1155
Rename package to lambdainline instead of lambdaInline
MaartenS11 Aug 9, 2023
bcab91e
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
MaartenS11 Aug 9, 2023
2710ff3
Rename lambda_locator package to lambdalocator
MaartenS11 Aug 9, 2023
49bd302
Added newline to Lambda class and removed some unneeded methods
MaartenS11 Aug 9, 2023
55b4f0e
Use result of lambdaUsageHandler.handle() directly instead of storing…
MaartenS11 Aug 9, 2023
7613999
This doesn't have to be a nested if
MaartenS11 Aug 9, 2023
ee3e3ab
Added more javadoc comments to the IterativeInstructionVisitor
MaartenS11 Aug 9, 2023
f1ba773
Added more javadoc strings and fixed a typo
MaartenS11 Aug 9, 2023
7d602c9
Renamed isSourceInstruction to isNotSourceInstruction
TimothyGeerkensGuardsquare Aug 9, 2023
4d18370
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
TimothyGeerkensGuardsquare Aug 9, 2023
2648338
Added more javadoc comments
MaartenS11 Aug 9, 2023
b431cb7
Apply suggestions from code review
MaartenS11 Aug 9, 2023
11f21f1
Use Optional in MethodLengthFinder so the programmer is made aware th…
MaartenS11 Aug 9, 2023
028e8c4
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
MaartenS11 Aug 9, 2023
0cc53cd
Return ProgramMethod instead of just Method
MaartenS11 Aug 9, 2023
a440b45
Use the name kotlin lambdas instead of static lambdas in the LambdaLo…
MaartenS11 Aug 9, 2023
7e652b3
Renamed constantInstruction and offset to be more precise also added …
MaartenS11 Aug 9, 2023
607bf86
Added more info about what "a lambda" really means
MaartenS11 Aug 10, 2023
f69fd14
Change some things to the FixedPointCodeAttributeVisitor API
MaartenS11 Aug 10, 2023
95b23ab
invokespecial is now considered a potential source for the sourcetracer
TimothyGeerkensGuardsquare Aug 10, 2023
c5bca4a
merge
TimothyGeerkensGuardsquare Aug 10, 2023
23d4efe
Use library and program class pools instead of passing AppView
MaartenS11 Aug 10, 2023
7cdac26
Merge branch 'lambda-inlining' of github.com:MaartenS11/proguard into…
MaartenS11 Aug 10, 2023
03ae5c6
Made one of the debug log messages info again because it was accident…
MaartenS11 Aug 10, 2023
ad8b5ff
Add large javadoc comment explaining all the steps taken when using t…
MaartenS11 Aug 10, 2023
26d9850
Added comment explaining why descriptor is changed to use unboxed types
MaartenS11 Aug 10, 2023
a5ebb5e
Move a lot of code from the BaseLambdaInliner into a private inner cl…
MaartenS11 Aug 10, 2023
71b63a4
Fixed incorrect comment mentioning method that no longer exists
MaartenS11 Aug 10, 2023
7da8e01
Disable replacing usages with aconst_null
MaartenS11 Aug 10, 2023
751cc74
Add -inlinekotlinlambdas command line option to enable and disable la…
MaartenS11 Aug 10, 2023
f4f84e4
Moved lambda inlining optimization from inline to lambdainline packag…
MaartenS11 Aug 10, 2023
9d080b2
Add CannotInlineException message to debug log
MaartenS11 Aug 10, 2023
b8fbcd0
Add ProGuard header to every class
MaartenS11 Aug 11, 2023
3a866fc
Added detection for the second null check method for parameters in Ko…
MaartenS11 Aug 11, 2023
a910309
Added WIP README file to the lambdainline package
MaartenS11 Aug 11, 2023
0cf13a7
Fix example and explain more about the cast removal/boxing/unboxing
MaartenS11 Aug 11, 2023
1ba33ba
Mention a bit more which classes are used to do which operations
MaartenS11 Aug 11, 2023
ceea1b6
Be more specific when using the ClassReferenceInitializer, not all re…
MaartenS11 Aug 12, 2023
93d3bb6
Only re-run the ClassReferenceInitializer on the consuming class
MaartenS11 Aug 12, 2023
ac7f42d
Only run AccessFixer on consumingClass instead of the entire programC…
MaartenS11 Aug 12, 2023
370704b
Adjust conditional inlining attempt condition to use sum of consuming…
MaartenS11 Aug 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Removed some unused fields from the RecursiveInliner
  • Loading branch information
MaartenS11 committed Aug 7, 2023
commit 573995e59ddd14e8d6f25eb0b6a30ba57d599ad7
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ public void visitAnyMember(Clazz clazz, Member member) {

// Don't inline if the lamdba is passed to another method
try {
copiedConsumingMethod.accept(consumingClass, new RecursiveInliner(appView, consumingMethod, lambda));
copiedConsumingMethod.accept(consumingClass, new RecursiveInliner(consumingMethod));
} catch(CannotInlineException cie) {
MaartenS11 marked this conversation as resolved.
Show resolved Hide resolved
ClassEditor classEditor = new ClassEditor((ProgramClass) consumingClass);
classEditor.removeMethod(copiedConsumingMethod);
Expand Down
56 changes: 3 additions & 53 deletions base/src/main/java/proguard/optimize/inline/RecursiveInliner.java
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
package proguard.optimize.inline;

import proguard.optimize.inline.lambda_locator.LambdaLocator;
import proguard.AppView;
import proguard.classfile.*;
import proguard.classfile.attribute.Attribute;
import proguard.classfile.attribute.BootstrapMethodsAttribute;
Expand All @@ -10,60 +8,36 @@
import proguard.classfile.attribute.visitor.AttributeVisitor;
import proguard.classfile.constant.*;
import proguard.classfile.constant.visitor.ConstantVisitor;
import proguard.classfile.editor.ClassEditor;
import proguard.classfile.editor.CodeAttributeEditor;
import proguard.classfile.editor.ConstantPoolEditor;
import proguard.classfile.instruction.ConstantInstruction;
import proguard.classfile.instruction.Instruction;
import proguard.classfile.instruction.InstructionFactory;
import proguard.classfile.instruction.VariableInstruction;
import proguard.classfile.instruction.visitor.InstructionOpCodeFilter;
import proguard.classfile.instruction.visitor.InstructionVisitor;
import proguard.classfile.util.ClassReferenceInitializer;
import proguard.classfile.util.ClassUtil;
import proguard.classfile.visitor.ClassPrinter;
import proguard.classfile.visitor.MemberVisitor;
import proguard.evaluation.PartialEvaluator;
import proguard.evaluation.TracedStack;
import proguard.optimize.inline.lambda_locator.Lambda;

/**
* Recursively inline functions that make use of the lambda parameter in the arguments of the current function.
* The first step, is finding out who actually uses our lambda parameter, we do this using the partial evaluator.
*/
public class RecursiveInliner implements AttributeVisitor, InstructionVisitor, MemberVisitor, ConstantVisitor {
private final AppView appView;
private Clazz consumingClazz;
private Method copiedConsumingMethod;
private final Method originalConsumingMethod;
private final boolean isStatic;
private final Lambda lambda;
private final boolean enableRecursiveMerging;
private PartialEvaluator partialEvaluator;
private final CodeAttributeEditor codeAttributeEditor;
private Clazz referencedClass;
private Method referencedMethod;
private int callOffset;
private boolean changed = false;
private final PartialEvaluator partialEvaluator;

/**
* @param originalConsumingMethod The original consuming method reference, this is used to detect recursion.
*/
public RecursiveInliner(AppView appView, Method originalConsumingMethod, Lambda lambda, boolean enableRecursiveMerging) {
this.appView = appView;
public RecursiveInliner(Method originalConsumingMethod) {
this.consumingClazz = null;
this.copiedConsumingMethod = null;
this.originalConsumingMethod = originalConsumingMethod;
this.isStatic = (originalConsumingMethod.getAccessFlags() & AccessConstants.STATIC) != 0;
this.lambda = lambda;
this.enableRecursiveMerging = enableRecursiveMerging;
this.partialEvaluator = new PartialEvaluator();
this.codeAttributeEditor = new CodeAttributeEditor();
this.referencedMethod = null;
}

public RecursiveInliner(AppView appView, Method originalConsumingMethod, Lambda lambda) {
this(appView, originalConsumingMethod, lambda, false);
}

@Override
Expand Down Expand Up @@ -120,7 +94,6 @@ public void visitMethodrefConstant(Clazz clazz, MethodrefConstant methodrefConst
// Note that the instruction is only volatile.
Instruction instruction = InstructionFactory.create(codeAttribute.code, offset);
int instructionLength = instruction.length(offset);
changed = false;
instruction.accept(clazz, method, codeAttribute, offset, new InstructionOpCodeFilter(
new int[] {
Instruction.OP_INVOKESTATIC,
Expand All @@ -132,8 +105,6 @@ public void visitMethodrefConstant(Clazz clazz, MethodrefConstant methodrefConst
new InstructionVisitor() {
@Override
public void visitConstantInstruction(Clazz clazz, Method method, CodeAttribute codeAttribute, int offset, ConstantInstruction constantInstruction) {
if (changed)
return;
System.out.println("At " + constantInstruction);
TracedStack tracedStack = partialEvaluator.getStackBefore(offset);

Expand Down Expand Up @@ -168,13 +139,9 @@ public void visitBootstrapMethodsAttribute(Clazz clazz, BootstrapMethodsAttribut
if (bootstrapMethodHandle.getClassName(clazz).equals("java/lang/invoke/LambdaMetafactory"))
{
MethodHandleConstant methodHandleConstant = (MethodHandleConstant) ((ProgramClass) clazz).getConstant(bootstrapMethodInfo.u2methodArguments[1]);
referencedMethod = new RefMethodFinder(clazz).findReferencedMethod(methodHandleConstant.u2referenceIndex);
referencedClass = clazz;
Method referencedMethod = new RefMethodFinder(clazz).findReferencedMethod(methodHandleConstant.u2referenceIndex);
System.out.println(referencedMethod);

if (changed)
return;

String methodDescriptor = referencedMethod.getDescriptor(clazz);
int argCount = ClassUtil.internalMethodParameterCount(methodDescriptor);
boolean referencedMethodStatic = (referencedMethod.getAccessFlags() & AccessConstants.STATIC) != 0;
Expand All @@ -186,11 +153,7 @@ public void visitBootstrapMethodsAttribute(Clazz clazz, BootstrapMethodsAttribut
codeAttribute.accept(clazz, method, new ClassPrinter());
int stackAdjustedIndex = ClassUtil.internalMethodVariableIndex(methodDescriptor, referencedMethodStatic, argIndex);
int traceOffset = tracedStack.getBottomActualProducerValue(tracedStack.size() - ClassUtil.internalMethodVariableIndex(methodDescriptor, referencedMethodStatic, argCount) + stackAdjustedIndex).instructionOffsetValue().instructionOffset(0);
callOffset = offset;
codeAttribute.instructionAccept(clazz, method, traceOffset, RecursiveInliner.this);
if (changed) {
break;
}
}
}
});
Expand All @@ -204,9 +167,6 @@ public void visitInterfaceMethodrefConstant(Clazz clazz, InterfaceMethodrefConst
}

public void handle(AnyMethodrefConstant methodrefConstant) {
if (changed)
return;

if (methodrefConstant.getClassName(clazz).equals("kotlin/jvm/internal/Intrinsics") &&
methodrefConstant.getName(clazz).equals("checkNotNullParameter") &&
methodrefConstant.getType(clazz).equals("(Ljava/lang/Object;Ljava/lang/String;)V"))
Expand All @@ -223,23 +183,13 @@ public void handle(AnyMethodrefConstant methodrefConstant) {
codeAttribute.accept(clazz, method, new ClassPrinter());
int stackAdjustedIndex = ClassUtil.internalMethodVariableIndex(methodDescriptor, referencedMethodStatic, argIndex);
int traceOffset = tracedStack.getBottomActualProducerValue(tracedStack.size() - ClassUtil.internalMethodVariableIndex(methodDescriptor, referencedMethodStatic, argCount) + stackAdjustedIndex).instructionOffsetValue().instructionOffset(0);
referencedMethod = methodrefConstant.referencedMethod;
referencedClass = methodrefConstant.referencedClass;
callOffset = offset;
codeAttribute.instructionAccept(clazz, method, traceOffset, RecursiveInliner.this);
if (changed) {
break;
}
}
}
});
}
}
));
if (changed) {
offset = 0;
continue;
}
offset += instructionLength;
}
}
Expand Down