Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
174fcd2
Implemented draft of RMMR algorithm.
RamSaw Mar 28, 2018
6f61018
Made several changes: corrected wrong behaviour because of missing br…
RamSaw Apr 11, 2018
0fd31dd
Corrected test: it ran all algorithms, but assertions for RMMR algo a…
RamSaw Apr 11, 2018
bc85cad
Corrected marks from pull request.
RamSaw Apr 15, 2018
df7b1e0
Added tests for MovieRentalStore example.
RamSaw Apr 24, 2018
08536b2
Added javadocs and made code clean up.
RamSaw Apr 24, 2018
538ff85
Changed to run parallel. Parallel executing is not comfortable for de…
RamSaw Apr 24, 2018
59b19a2
Made RMMR constructor public. IDEA says it can be package-private but…
RamSaw Apr 24, 2018
1fce0bc
Made RmmrEntitySearcher more consistent and turned to qualified names.
RamSaw May 21, 2018
79ebe6b
Moved 0.01 hardcoded constant to static final constant.
RamSaw May 21, 2018
22ac2b6
Removed new expressions, added Builders, Utils, Factory cases, change…
RamSaw May 29, 2018
e227e2b
Added contextual similarity calculation. Very raw version. TODO: opti…
RamSaw Jul 6, 2018
2d0ddd0
Fixed null pointer exception by change get to getOrDefault. Also adde…
RamSaw Jul 6, 2018
e9f71fd
Changed to multisets and all maps from entity moved to entity class t…
RamSaw Jul 6, 2018
00333f5
Deleted redundant documents sustaining. Now it consists of two refere…
RamSaw Jul 9, 2018
2c4d416
Added support of configuration of algorithm.
RamSaw Jul 9, 2018
c1379f4
Added stemming.
RamSaw Jul 9, 2018
2400fb5
Reverted ArchitectureReloaded.iml file.
RamSaw Jul 9, 2018
66c2b2c
fix conflicts commit
RamSaw Jul 9, 2018
826ee53
Added jar for stemming
RamSaw Jul 9, 2018
83168b2
Removed qualified name error fix because tests were not passing.
RamSaw Jul 9, 2018
9eef351
Merge branch 'RMMR_implementation' of https://github.com/ml-in-progra…
RamSaw Jul 9, 2018
42a7471
Erased bug with qualified names and added excluding short names like …
RamSaw Jul 9, 2018
08740ec
Added tests for RMMR and added support for setting up or not field re…
RamSaw Jul 9, 2018
ebe7152
Speeded up algorithm by adding coordinates that are only not null. Th…
RamSaw Jul 10, 2018
b9e8966
Fixed tests. Failing test were uncommented by mistake.
RamSaw Jul 10, 2018
28153b7
Algorithm was optimized more, now bag finder class deleted and its lo…
RamSaw Jul 10, 2018
d2f1015
Found mistake by dividing on zero. Fixed and as a consequence move me…
RamSaw Jul 10, 2018
c0503ef
RMMR was set up to pass more test cases. On real project output is no…
RamSaw Jul 10, 2018
9f884cd
Ignored tests in RmmrDistancesTest and RmmrEntitySearcherTest. Reason…
RamSaw Jul 10, 2018
5cd9df9
Fixed tests failure. Ignored tests in RmmrDistancesTest and RmmrEntit…
RamSaw Jul 10, 2018
82ade8e
Added support for method references: ClassA::methodInClassA.
RamSaw Jul 10, 2018
bc238ca
Added support for class references. For example call of static method…
RamSaw Jul 10, 2018
a342283
Added failure explanations for all tests which fail.
RamSaw Jul 10, 2018
f15f724
Corrected weights for conceptual and contextual similarity.
RamSaw Jul 10, 2018
9bd97d4
Corrected accuracy formula - now output is a little bit better.
RamSaw Jul 11, 2018
fd94e63
Made clean up of code, code prepared for review. Deleted old test data.
RamSaw Jul 11, 2018
3d86756
Corrected accuracy formula to fit other algorithms. Before that accur…
RamSaw Jul 11, 2018
afa2096
Fixes #50
RamSaw Jul 11, 2018
9fc5780
Fixed test failure.
RamSaw Jul 11, 2018
ac4cf63
Reverted bug fix, because it introduced new bug.
RamSaw Jul 13, 2018
bd370e9
Merge remote-tracking branch 'origin/master' into RMMR_implementation
RamSaw Jul 16, 2018
c183397
Merge remote-tracking branch 'origin/master' into RMMR_implementation
RamSaw Jul 16, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .idea/copyright/ArchitectureReloaded.xml

This file was deleted.

7 changes: 0 additions & 7 deletions .idea/copyright/profiles_settings.xml

This file was deleted.

1 change: 1 addition & 0 deletions ArchitectureReloaded.iml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
<content url="file://$MODULE_DIR$">
<excludeFolder url="file://$MODULE_DIR$/.gradle" />
<excludeFolder url="file://$MODULE_DIR$/build" />
<excludeFolder url="file://$MODULE_DIR$/out" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
Expand Down
14 changes: 4 additions & 10 deletions openapi/openapi.iml
Original file line number Diff line number Diff line change
@@ -1,15 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<module relativePaths="true" type="JAVA_MODULE" version="4">
<component name="NewModuleRootManager" inherit-compiler-output="false">
<output url="file://$MODULE_DIR$/build/classes" />
<output-test url="file://$MODULE_DIR$/build/testclasses" />
<module external.linked.project.id="openapi" external.linked.project.path="$MODULE_DIR$" external.root.project.path="$MODULE_DIR$" external.system.id="GRADLE" type="JAVA_MODULE" version="4">
<component name="NewModuleRootManager" inherit-compiler-output="true">
<exclude-output />
<content url="file://$MODULE_DIR$">
<sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
<excludeFolder url="file://$MODULE_DIR$/build" />
</content>
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>

</module>
131 changes: 131 additions & 0 deletions src/main/java/org/ml_methods_group/algorithm/RMMR.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
package org.ml_methods_group.algorithm;

import org.apache.log4j.Logger;
import org.ml_methods_group.algorithm.entity.ClassEntity;
import org.ml_methods_group.algorithm.entity.EntitySearchResult;
import org.ml_methods_group.algorithm.entity.MethodEntity;
import org.ml_methods_group.config.Logging;
import org.ml_methods_group.utils.AlgorithmsUtil;

import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class RMMR extends Algorithm {
public static final String NAME = "RMMR";

private static final Logger LOGGER = Logging.getLogger(MRI.class);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected.


private final Map<ClassEntity, Set<MethodEntity>> methodsByClass = new HashMap<>();
private final List<MethodEntity> units = new ArrayList<>();
private final List<ClassEntity> classEntities = new ArrayList<>();
private final AtomicInteger progressCount = new AtomicInteger();
private ExecutionContext context;

public RMMR() {
super(NAME, true);
}

@Override
protected List<Refactoring> calculateRefactorings(ExecutionContext context, boolean enableFieldRefactorings) throws Exception {
if (enableFieldRefactorings) {
// TODO: write to LOGGER or throw Exception? Change UI: disable field checkbox if only RMMR is chosen.
LOGGER.error("Field refactorings are not supported",
new UnsupportedOperationException("Field refactorings are not supported"));
}
this.context = context;
init();
return runParallel(units, context, ArrayList::new, this::findRefactoring, AlgorithmsUtil::combineLists);
}

private void init() {
final EntitySearchResult entities = context.getEntities();
LOGGER.info("Init RMMR");
units.clear();
classEntities.clear();

classEntities.addAll(entities.getClasses());
units.addAll(entities.getMethods());
progressCount.set(0);

entities.getMethods().forEach(methodEntity -> {
List<ClassEntity> methodClass = entities.getClasses().stream()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

entities -> List of Entity
classEntities -> List of ClassEntity
methodClass -> ???

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to methodClassEntity. This list must be size of 1 and consist of class entity in which method is located.

.filter(classEntity -> methodEntity.getClassName().equals(classEntity.getName()))
.collect(Collectors.toList());
if (methodClass.size() != 1) {
LOGGER.error("Found more than 1 class that has this method");
}
methodsByClass.computeIfAbsent(methodClass.get(0), anyKey -> new HashSet<>()).add(methodEntity);
});
}

private List<Refactoring> findRefactoring(MethodEntity entity, List<Refactoring> accumulator) {
reportProgress((double) progressCount.incrementAndGet() / units.size(), context);
context.checkCanceled();
if (!entity.isMovable() || classEntities.size() < 2) {
return accumulator;
}
double minDistance = Double.POSITIVE_INFINITY;
double difference = Double.POSITIVE_INFINITY;
ClassEntity targetClass = null;
for (final ClassEntity classEntity : classEntities) {
final double distance = getDistance(entity, classEntity);
if (distance < minDistance) {
difference = minDistance - distance;
minDistance = distance;
targetClass = classEntity;
} else if (distance - minDistance < difference) {
difference = distance - minDistance;
}
}

if (targetClass == null) {
LOGGER.warn("targetClass is null for " + entity.getName());
return accumulator;
}
final String targetClassName = targetClass.getName();
double accuracy = (1 - minDistance) * difference; // TODO: Maybe consider amount of entities?
if (!targetClassName.equals(entity.getClassName())) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a bit odd check looking on the code. if CLASS whose method is being analyzed there is taken into account when calculating the distance -- algorithm above might give a closer metric and you'll lose correct result. Probably more correct approach is to perform this check before the call to getDistance. Please correct me if I'm wrong.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it seems to me to be correct. We measure distance between CLASS and method, and distances between other classes and method. getDistance considers if it is calculating distance between CLASS and method so it will be counted as described in the article (without measuring method with itself which would be always 0 distance). And we suggest refactoring if there is less distance than with CLASS.

accumulator.add(new Refactoring(entity.getName(), targetClassName, accuracy, entity.isField()));
}
return accumulator;
}

private double getDistance(MethodEntity methodEntity, ClassEntity classEntity) {
int number = 0;
double sumOfDistances = 0;

if (methodsByClass.containsKey(classEntity)) {
for (MethodEntity methodEntityInClass : methodsByClass.get(classEntity)) {
if (!methodEntity.equals(methodEntityInClass)) {
sumOfDistances += getDistance(methodEntity, methodEntityInClass);
number++;
}
}
}

return number == 0 ? 1 : sumOfDistances / number;
}

private double getDistance(MethodEntity methodEntity1, MethodEntity methodEntity2) {
// TODO: Maybe add to methodEntity2 source class where it is located?
int sizeOfIntersection = intersection(methodEntity1.getRelevantProperties().getClasses(),
methodEntity2.getRelevantProperties().getClasses()).size();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might like to extract 2 variables there and these 4 lines will become more readable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected.

int sizeOfUnion = union(methodEntity1.getRelevantProperties().getClasses(),
methodEntity2.getRelevantProperties().getClasses()).size();
return sizeOfIntersection == 0 ? 1 : 1 - (double) sizeOfIntersection / sizeOfUnion;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't hesitate to use parenthesis around conditional statements

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understood you correctly, you suggested to write so: (sizeOfIntersection == 0). If yes, then I've corrected it.

}

private <T> Set<T> intersection(Set<T> set1, Set<T> set2) {
Set<T> intersection = new HashSet<>(set1);
intersection.retainAll(set2);
return intersection;
}

private <T> Set<T> union(Set<T> set1, Set<T> set2) {
Set<T> union = new HashSet<>(set1);
union.addAll(set2);
return union;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ private int getWeightedSize(Map<?, Integer> m) {
return m.values().stream().mapToInt(Integer::valueOf).sum();
}

int sizeOfIntersection(RelevantProperties properties) {
public int sizeOfIntersection(RelevantProperties properties) {
int result = 0;

final BinaryOperator<Integer> bop = Math::min;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
package org.ml_methods_group.algorithm.entity;

import com.intellij.analysis.AnalysisScope;
import com.intellij.openapi.progress.EmptyProgressIndicator;
import com.intellij.openapi.progress.ProgressIndicator;
import com.intellij.openapi.progress.ProgressManager;
import com.intellij.psi.*;
import org.apache.log4j.Logger;
import org.jetbrains.annotations.Contract;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.ml_methods_group.algorithm.properties.finder_strategy.FinderStrategy;
import org.ml_methods_group.algorithm.properties.finder_strategy.RmmrStrategy;
import org.ml_methods_group.config.Logging;

import java.util.*;

public class RmmrEntitySearcher {
private static final Logger LOGGER = Logging.getLogger(EntitySearcher.class);

private final Map<String, PsiClass> classForName = new HashMap<>();
private final Map<PsiMethod, MethodEntity> entities = new HashMap<>();
private final Map<PsiClass, ClassEntity> classEntities = new HashMap<>();
private final AnalysisScope scope;
private final long startTime;
private final FinderStrategy strategy;
private final ProgressIndicator indicator;

private RmmrEntitySearcher(AnalysisScope scope) {
this.scope = scope;
strategy = RmmrStrategy.getInstance();
startTime = System.currentTimeMillis();
if (ProgressManager.getInstance().hasProgressIndicator()) {
indicator = ProgressManager.getInstance().getProgressIndicator();
} else {
indicator = new EmptyProgressIndicator();
}
}

@NotNull
public static EntitySearchResult analyze(AnalysisScope scope) {
final RmmrEntitySearcher finder = new RmmrEntitySearcher(scope);
return finder.runCalculations();
}

@NotNull
private EntitySearchResult runCalculations() {
indicator.pushState();
indicator.setText("Searching entities");
indicator.setIndeterminate(true);
LOGGER.info("Indexing entities...");
scope.accept(new UnitsFinder());
indicator.setIndeterminate(false);
LOGGER.info("Calculating properties...");
indicator.setText("Calculating properties");
scope.accept(new PropertiesCalculator());
indicator.popState();
return prepareResult();
}

@NotNull
private EntitySearchResult prepareResult() {
LOGGER.info("Preparing results...");
final List<ClassEntity> classes = new ArrayList<>();
final List<MethodEntity> methods = new ArrayList<>();
for (MethodEntity methodEntity : entities.values()) {
indicator.checkCanceled();
methods.add(methodEntity);
}
for (ClassEntity classEntity : classEntities.values()) {
indicator.checkCanceled();
classes.add(classEntity);
}
LOGGER.info("Properties calculated");
LOGGER.info("Generated " + classes.size() + " class entities");
LOGGER.info("Generated " + methods.size() + " method entities");
LOGGER.info("Generated " + 0 + " field entities. Fields are not supported.");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this line make any sense?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know. I report that this algorithm does not work with fields. I think it serves for consistency, because log information in EntitySearcher is structured the same.

return new EntitySearchResult(classes, methods, Collections.emptyList(), System.currentTimeMillis() - startTime);
}


private class UnitsFinder extends JavaRecursiveElementVisitor {
@Override
public void visitFile(PsiFile file) {
indicator.checkCanceled();
if (strategy.acceptFile(file)) {
LOGGER.info("Indexing " + file.getName());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't look consistent to methods below

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as well as info level for file visits is too high.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made consistent. Not sure that I understood correctly about too high level. Does it mean that this information is not so important to be logged in info? I peeped this code in EntitySearcher implementation, it is exactly like there. If it is really too high level then we need to correct it in EntitySearcher also.

super.visitFile(file);
}
}

@Override
public void visitClass(PsiClass aClass) {
indicator.checkCanceled();
//classForName.put(getHumanReadableName(aClass), aClass);
//TODO: maybe qualified name? Otherwise name collision may occur.
classForName.put(aClass.getName(), aClass);
if (!strategy.acceptClass(aClass)) {
return;
}
classEntities.put(aClass, new ClassEntity(aClass));
super.visitClass(aClass);
}

@Override
public void visitMethod(PsiMethod method) {
indicator.checkCanceled();
if (!strategy.acceptMethod(method)) {
return;
}
entities.put(method, new MethodEntity(method));
super.visitMethod(method);
}
}


// TODO: calculate properties for constructors? If yes, then we need to separate methods to check on refactoring (entities) and methods for calculating metric (to gather properties).
private class PropertiesCalculator extends JavaRecursiveElementVisitor {
private int propertiesCalculated = 0;

private MethodEntity currentMethod;

@Override
public void visitMethod(PsiMethod method) {
indicator.checkCanceled();
final MethodEntity methodEntity = entities.get(method);
if (methodEntity == null) {
super.visitMethod(method);
return;
}
final RelevantProperties methodProperties = methodEntity.getRelevantProperties();
if (currentMethod == null) {
currentMethod = methodEntity;
}

for (PsiParameter attribute : method.getParameterList().getParameters()) {
PsiType attributeType = attribute.getType();
if (attributeType instanceof PsiClassType) {
String className = ((PsiClassType) attributeType).getClassName();
if (isClassInScope(className)) {
methodProperties.addClass(classForName.get(className));
}
}
}
super.visitMethod(method);
if (currentMethod == methodEntity) {
currentMethod = null;
}
reportPropertiesCalculated();
}

@Override
public void visitNewExpression(PsiNewExpression expression) {
indicator.checkCanceled();
String className = null;
PsiType type = expression.getType();
if (type instanceof PsiClassType) {
className = ((PsiClassType) expression.getType()).getClassName();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(PsiClassType) type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected.

}
final PsiClass usedClass = classForName.get(className);
if (currentMethod != null && className != null && isClassInScope(usedClass)) {
currentMethod.getRelevantProperties().addClass(usedClass);
}
super.visitNewExpression(expression);
}

@Override
public void visitMethodCallExpression(PsiMethodCallExpression expression) {
// Do not find constructors. It does not consider them as method calls.
indicator.checkCanceled();
final PsiMethod called = expression.resolveMethod();
final PsiClass usedClass = called != null ? called.getContainingClass() : null;
if (currentMethod != null && called != null && isClassInScope(usedClass)) {
currentMethod.getRelevantProperties().addClass(usedClass);
}
super.visitMethodCallExpression(expression);
}

private void reportPropertiesCalculated() {
propertiesCalculated++;
if (indicator != null) {
indicator.setFraction((double) propertiesCalculated / entities.size());
}
}

@Contract(pure = true)
private boolean isClassInScope(String aClass) {
return classForName.containsKey(aClass);
}

@Contract("null -> false")
private boolean isClassInScope(final @Nullable PsiClass aClass) {
return aClass != null && classForName.containsKey(aClass.getName());
}
}
}
Loading