-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speed up Refaster
bug checker
#261
Draft
Stephan202
wants to merge
58
commits into
master
Choose a base branch
from
sschroevers/refaster-speed-up
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit
Hold shift + click to select a range
69386f0
Emit website link along with Refaster refactor suggestions
Stephan202 84061fc
More extensible approach
Stephan202 827880b
WIP: Some plumbing for annotation support
Stephan202 0f44844
Another round
Stephan202 0e46f9c
Tweaks
Stephan202 914d30a
Better annotation support, simplify setup
Stephan202 458fb99
Flag why build currently doesn't fail, while it should.
Stephan202 db8cf3c
Improve logic and test coverage
Stephan202 abb6cea
Properly document URL placeholder usage
Stephan202 cf772c4
Expand test coverage
Stephan202 d26bc18
Flag classpath issue
Stephan202 01b7e5b
Improve text and minor improvements
rickie 3482641
Use `@AutoValue` for the `AnnotatedCompositeCodeTransformer`
rickie 12d09ad
Support `AllSuggestionsAsWarnings` and add a suggestion
rickie 74d6c9a
Tweak
rickie c4b6a5f
Suggestions, introduce `ErrorProneFork`
Stephan202 d899a8a
Also this
Stephan202 20d194b
Clarify how the default Refaster hit severity comes to be
Stephan202 ad1c98d
Align documentation of reported description names by the Refaster check
Badbond 50443d1
Suggestion
Badbond 7e49b08
Pass null for urlLink to Description.Builder
Badbond 594f51c
Tweaks
Stephan202 e6caceb
Move `AnnotatedCompositeCodeTransformer` and `ErrorProneFork` to `ref…
rickie b2ef631
`s/information/content/`
rickie 8bd88bb
Improve Javadoc `AnnotatedCompositeCodeTransformer`
rickie 302e20b
Revert changes in `OptionalTemplates`
rickie 32300ff
Tweak `AnnotatedCompositeCodeTransformer` Javadoc
Badbond d21ac59
Apply `StringJoin` suggestion
Stephan202 0484762
Suggestions
Stephan202 1624ebf
WIP: Speed up `Refaster` check
Stephan202 ee74279
Compatibility with stock Error Prone
Stephan202 344f4e4
Initial copy over of the improved algorithm
rickie 6739cb4
Improve code and algorithm for Refaster
rickie 72ff8ae
Drop unnecessary `@AutoService` annotation
rickie 39dc9aa
Add XXXs
rickie 2a93011
Merge `RefasterRuleSelector` type hierarchy
Stephan202 1c5077f
Create selector only once per `Refaster` instantiation
Stephan202 2c137c0
Move all `RefasterRuleSelector` construction logic into the relevant …
Stephan202 c3a5106
Push sorting requirements into `Node`, minimize tree, add tests
Stephan202 9151287
Suggestion
rickie a27d635
Extract the `TreeScanner`s to their own classes
rickie 2003bd2
Introduce `getClass` method to deduplicate
rickie 7889148
Reorder methods in `RefasterIntrospection`
rickie afdcf52
Apply some suggestions
Stephan202 ab03739
Optimize code, introduce benchmark, simplify test
Stephan202 457a8e2
Comment style, explain both performance-only pieces of code
Stephan202 63ad14e
Fix typo
rickie 0cf891c
Suggestions
ibabiankou 2cb4bd0
Suggestions
Stephan202 c34fa4d
Fix typo
rickie d59b626
Merge remote-tracking branch 'origin/sschroevers/refaster-custom-urls…
Stephan202 5ba7075
Introduce `AnnotatedCompositeCodeTransformerTest`
Stephan202 0157692
Merge remote-tracking branch 'origin/sschroevers/refaster-custom-urls…
Stephan202 72f8881
Suggestions
Stephan202 77bc107
Merge branch 'master' into sschroevers/refaster-speed-up
Stephan202 2687322
Merge branch master into sschroevers/refaster-speed-up
rickie 424b520
Merge branch 'master' into sschroevers/refaster-speed-up
rickie 8d99611
Post-merge-fix
rickie File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Suggestions
- Loading branch information
commit 0cf891c87b3249726b0f7592fae5f9ce48b226c5
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,8 +7,6 @@ | |
import com.google.common.collect.ImmutableCollection; | ||
import com.google.common.collect.ImmutableList; | ||
import com.google.common.collect.ImmutableSet; | ||
import com.google.errorprone.CodeTransformer; | ||
import com.google.errorprone.CompositeCodeTransformer; | ||
import com.google.errorprone.refaster.BlockTemplate; | ||
import com.google.errorprone.refaster.ExpressionTemplate; | ||
import com.google.errorprone.refaster.RefasterRule; | ||
|
@@ -39,7 +37,6 @@ | |
import java.util.IdentityHashMap; | ||
import java.util.List; | ||
import java.util.Set; | ||
import java.util.function.Consumer; | ||
import javax.annotation.Nullable; | ||
|
||
// XXX: Add some examples of which source files would match what templates in the tree. | ||
|
@@ -60,8 +57,8 @@ | |
* <li>Extract all identifiers from the {@link CompilationUnitTree} and sort them | ||
* lexicographically. | ||
* <li>Traverse the tree based on the identifiers from the {@link CompilationUnitTree}. Every node | ||
* can contain Refaster templates. Once a node is we found a candidate Refaster template and | ||
* will therefore be added to the list of candidates. | ||
* can contain Refaster templates. Once a node is we found a candidate Refaster template that | ||
* might match some code and will therefore be added to the list of candidates. | ||
* </ol> | ||
* | ||
* <p>This is an example to explain the algorithm. Consider the templates with identifiers; {@code | ||
|
@@ -77,14 +74,14 @@ | |
* └── D -- T3 | ||
* }</pre> | ||
* | ||
* <p>The tree is traversed based on the identifiers in the {@link CompilationUnitTree}. When a leaf | ||
* contains a template and is reached, we can be certain that the identifiers from the {@link | ||
* <p>The tree is traversed based on the identifiers in the {@link CompilationUnitTree}. When a node | ||
* containing a template is reached, we can be certain that the identifiers from the {@link | ||
* BeforeTemplate} are at least present in the {@link CompilationUnitTree}. | ||
* | ||
* <p>Since the identifiers are sorted, we can prune parts of the {@link Node tree} while we are | ||
* <p>Since the identifiers are sorted, we can skip parts of the {@link Node tree} while we are | ||
* traversing it. Instead of trying to match all Refaster templates against every expression in a | ||
* {@link CompilationUnitTree} we now only return a subset of the templates that at least have a | ||
* chance of matching. As a result, the performance of Refaster significantly increases. | ||
* {@link CompilationUnitTree} we now only matching a subset of the templates that at least have a | ||
* chance of matching. As a result, the performance of Refaster increases significantly. | ||
*/ | ||
final class RefasterRuleSelector { | ||
private final Node<RefasterRule<?, ?>> treeRules; | ||
|
@@ -93,36 +90,12 @@ private RefasterRuleSelector(Node<RefasterRule<?, ?>> treeRules) { | |
this.treeRules = treeRules; | ||
} | ||
|
||
/** | ||
* Instantiates a new {@link RefasterRuleSelector} backed by the {@link RefasterRule}s referenced | ||
* by the given collection of {@link CodeTransformer}s. | ||
*/ | ||
static RefasterRuleSelector create(ImmutableCollection<CodeTransformer> transformers) { | ||
List<RefasterRule<?, ?>> refasterRules = new ArrayList<>(); | ||
collectRefasterRules(transformers, refasterRules::add); | ||
/** Instantiates a new {@link RefasterRuleSelector} backed by the given {@link RefasterRule}s. */ | ||
static RefasterRuleSelector create(ImmutableCollection<RefasterRule<?, ?>> refasterRules) { | ||
return new RefasterRuleSelector( | ||
Node.create(refasterRules, RefasterRuleSelector::extractTemplateIdentifiers)); | ||
} | ||
|
||
private static void collectRefasterRules( | ||
ImmutableCollection<CodeTransformer> transformers, Consumer<RefasterRule<?, ?>> sink) { | ||
for (CodeTransformer t : transformers) { | ||
collectRefasterRules(t, sink); | ||
} | ||
} | ||
|
||
private static void collectRefasterRules( | ||
CodeTransformer transformer, Consumer<RefasterRule<?, ?>> sink) { | ||
if (transformer instanceof RefasterRule) { | ||
sink.accept((RefasterRule<?, ?>) transformer); | ||
} else if (transformer instanceof CompositeCodeTransformer) { | ||
collectRefasterRules(((CompositeCodeTransformer) transformer).transformers(), sink); | ||
} else { | ||
throw new IllegalStateException( | ||
String.format("Can't handle `CodeTransformer` of type '%s'", transformer.getClass())); | ||
} | ||
} | ||
|
||
/** | ||
* Retrieve a set of Refaster templates that can possibly match based on a {@link | ||
* CompilationUnitTree}. | ||
|
@@ -135,7 +108,6 @@ private static void collectRefasterRules( | |
Set<RefasterRule<?, ?>> selectCandidateRules(CompilationUnitTree tree) { | ||
Set<RefasterRule<?, ?>> candidateRules = newSetFromMap(new IdentityHashMap<>()); | ||
treeRules.collectReachableValues(extractSourceIdentifiers(tree), candidateRules::add); | ||
|
||
return candidateRules; | ||
} | ||
|
||
|
@@ -166,17 +138,13 @@ private static ImmutableSet<ImmutableSet<String>> extractTemplateIdentifiers( | |
ImmutableList<? extends Tree> trees) { | ||
List<Set<String>> identifierCombinations = new ArrayList<>(); | ||
identifierCombinations.add(new HashSet<>()); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we need to add an empty set here manually? I think it is worth adding a comment. |
||
|
||
new TemplateIdentifierExtractor().scan(trees, identifierCombinations); | ||
|
||
TemplateIdentifierExtractor.INSTANCE.scan(trees, identifierCombinations); | ||
return identifierCombinations.stream().map(ImmutableSet::copyOf).collect(toImmutableSet()); | ||
} | ||
|
||
private static Set<String> extractSourceIdentifiers(Tree tree) { | ||
Set<String> identifiers = new HashSet<>(); | ||
|
||
new SourceIdentifierExtractor().scan(tree, identifiers); | ||
|
||
SourceIdentifierExtractor.INSTANCE.scan(tree, identifiers); | ||
return identifiers; | ||
} | ||
|
||
|
@@ -354,6 +322,8 @@ private static <T> T invokeMethod(Method method, Object instance) { | |
} | ||
|
||
private static class TemplateIdentifierExtractor extends TreeScanner<Void, List<Set<String>>> { | ||
private static final TemplateIdentifierExtractor INSTANCE = new TemplateIdentifierExtractor(); | ||
|
||
@Nullable | ||
@Override | ||
public Void visitIdentifier(IdentifierTree node, List<Set<String>> identifierCombinations) { | ||
|
@@ -434,7 +404,8 @@ public Void visitBinary(BinaryTree node, List<Set<String>> identifierCombination | |
|
||
private static void registerOperator( | ||
ExpressionTree node, List<Set<String>> identifierCombinations) { | ||
identifierCombinations.forEach(ids -> ids.add(treeKindToString(node.getKind()))); | ||
String id = treeKindToString(node.getKind()); | ||
identifierCombinations.forEach(ids -> ids.add(id)); | ||
} | ||
|
||
@Nullable | ||
|
@@ -462,6 +433,8 @@ private static List<Set<String>> copy(List<Set<String>> identifierCombinations) | |
} | ||
|
||
private static class SourceIdentifierExtractor extends TreeScanner<Void, Set<String>> { | ||
private static final SourceIdentifierExtractor INSTANCE = new SourceIdentifierExtractor(); | ||
|
||
@Nullable | ||
@Override | ||
public Void visitPackage(PackageTree node, Set<String> identifiers) { | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ibabiankou I agree that with the PR as-is it was an improvement to move this
RefasterRule
extraction logic here, but the changes in #255 make theCodeTransformer
->ImmutableSet<ImmutableSet<String>>
transformation less straightforward, so I'll move it back. Of course, idea on how to better do this are welcome!