Report dead entry points #187

yiming-tang-cs · 2018-03-08T20:57:01Z

No description provided.

yiming-tang-cs · 2018-03-08T21:04:16Z

I did not change the set of entry points because I think the entry points should be consistent with the entry points which are used to create call graph. Hence, I just report the dead entry points for now.

yiming-tang-cs · 2018-03-09T02:40:26Z

When the tool evaluates a project, it generates a set of dead entry points and prints those entry points into a txt file named dead_entry_points.txt.

I've added 3 methods:

pruneEntryPoints(): the main methods to report dead entry points
isReachable: given an entry point, check whether there is a stream creation node which is reachable from the entry point
reportDeadEntryPoints(): print all entry points into txt file.

yiming-tang-cs · 2018-03-09T16:01:24Z

I've added two new dead entry points in to streamql and tested streamql.

The dead_entry_points.txt:

deephacks.streamql.LocalDateTimeTest.<clinit>()V
deephacks.streamql.LocalDateTimeTest.<init>()V
deephacks.streamql.DurationTest.<init>()V
deephacks.streamql.PersonTest.<clinit>()V
deephacks.streamql.PeriodTest.<init>()V
deephacks.streamql.ZonedDateTimeTest.<clinit>()V
deephacks.streamql.ExceptionalTest.<init>()V
deephacks.streamql.EnumTest.<init>()V
deephacks.streamql.UriTest.<clinit>()V
deephacks.streamql.NumbersTest.<init>()V
deephacks.streamql.UriTest.<init>()V
deephacks.streamql.UuidTest.<init>()V
deephacks.streamql.FileTest.<clinit>()V
deephacks.streamql.EnumTest.<clinit>()V
deephacks.streamql.StringTest.<init>()V
deephacks.streamql.NullTest.<init>()V
deephacks.streamql.NumbersTest.<clinit>()V
deephacks.streamql.ComparatorsTest.<init>()V
deephacks.streamql.ComparatorsTest.ints()V
deephacks.streamql.PersonTest.<init>()V
deephacks.streamql.UuidTest.<clinit>()V
deephacks.streamql.ComparatorsTest.longs()V
deephacks.streamql.DurationTest.<clinit>()V
deephacks.streamql.FileTest.<init>()V
deephacks.streamql.BooleanTest.<init>()V
deephacks.streamql.ZoneOffsetTest.<init>()V
deephacks.streamql.PeriodTest.<clinit>()V
deephacks.streamql.ZoneOffsetTest.<clinit>()V
deephacks.streamql.ZonedDateTimeTest.<init>()V
deephacks.streamql.StringTest.<clinit>()V
deephacks.streamql.BooleanTest.<clinit>()V

deephacks.streamql.ComparatorsTest.ints()V and deephacks.streamql.ComparatorsTest.longs()V are two dead entry points that I added. Others are from constructors.

yiming-tang-cs · 2018-03-09T16:20:37Z

Project:

class A {

	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {}
}

Result:

q.A.<init>()V
q.A.mm()V

Project:

class A {

	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {n();}
}

Result:

q.A.<init>()V

This test case is interesting:
Project:

class A {
	@EntryPoint
	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {m();}
}

Result:

q.A.mm()V
q.A.n()V
q.A.<init>()V

khatchad · 2018-03-09T16:30:23Z

How are the results correct for your last example? There should be no dead entry points there. They all lead to methods instantiating streams.

khatchad · 2018-03-09T16:31:38Z

Also, ctors and static initializers are only dead iff all non-ctor and non-static initializers in the associated class are dead.

yiming-tang-cs · 2018-03-09T18:29:18Z

How are the results correct for your last example? There should be no dead entry points there. They all lead to methods instantiating streams.

It seems that the current strategy only detects one path from the any entry point to a specific stream creation node and ignore other paths. I do not think the logic of my code is wrong for now. I need to check whether my work is wrong or this issue is from difference of call graph. I will check it.later.

khatchad · 2018-03-09T18:53:49Z

It would be good to add such test cases to the refactoring test suite, even if the assertions are not in place. At least we can see the output.

yiming-tang-cs · 2018-03-12T14:35:11Z

How are the results correct for your last example? There should be no dead entry points there. They all lead to methods instantiating streams.

The results are very strange here.
For this example:

class A {

	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {m();}
}

, the result are q.A.mm()V or q.A.n()V. I wonder why the code just pick one path from any entry point to a specific entry point and ignore other paths.

yiming-tang-cs · 2018-03-12T14:42:57Z

It would be good to add such test cases to the refactoring test suite, even if the assertions are not in place. At least we can see the output.

You have mentioned to test reporting entry points instead of refactoring. I have looked at helper() method. It calls analyzer.analyze(). However, this method contains both reporting dead entry points and refactoring. How can I separate them?

Or I just need to remove the parameters of helper()?

And the output file will be overwritten again and again. Should I rename them to avoid overwriting.

yiming-tang-cs · 2018-03-12T15:16:25Z

How are the results correct for your last example? There should be no dead entry points there. They all lead to methods instantiating streams.

I guess the reason of it may be that one CGNode only binds one method in its call string context.
For the example below:

class A {

	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {m();}
}

, the CGNode of m() should have two call string contexts. Actually, it only has one. Sometime, it has CallStringContext: [ q.A.mm()V@1 ] and sometime it has CallStringContext: [ q.A.n()V@1 ].

Because of this, I cannot get all entry points for a specific stream creation node and I can just get at most one entry point for a specific stream creation node.

khatchad · 2018-03-12T16:18:49Z

#187 (comment) should be another issue as well.

yiming-tang-cs · 2018-03-12T16:30:50Z

I've changed the code snippet of getting a set of stream creation nodes. Then, the results are right.
The test case:

class A {

	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {m();}
}

, I get nothing in the txt file.

When the test case is:

class A {

	@EntryPoint
	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {m();}
}

, I also get 0 dead entry points.

When the test is

class A {

	@EntryPoint
	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {}
}

, the output is

q.A.mm()V

yiming-tang-cs · 2018-03-12T18:22:10Z

Each node represents a method IMethod in a context.
http://wala.sourceforge.net/wiki/index.php/UserGuide:CallGraph

I did not find any official words to describe edges. I think each edge means the predecessor calls the successor.

The test case:

class A {

	void m() {
		HashSet h1 = new HashSet();
		h1.stream().count();
	}
	
	@EntryPoint
	void n() {
		m();
	}
	
	@EntryPoint
	void mm() {m();}
}

The CallGraph class represents potentially context-sensitive call graphs.
https://github.com/wala/WALA/wiki/Call-Graph

Hence, there should be 2 nodes of m() in the call graph because m() has two different contexts.

Node 0:

Node 1:

I found the context of a CGNode only contains call string context:

Can I say call string context is used to distinguish different CGNodes for the same method?

Then, I found the CGNode Node: < Application, Lq/A, m()V > Context: CallStringContext: [ q.A.mm()V@1 ] is reachable from the only entry point node Node: < Application, Lq/A, mm()V > Context: CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@6 ] and the CGNode Node: < Application, Lq/A, m()V > Context: CallStringContext: [ q.A.n()V@1 ] is reachable from the only entry point node Node: < Application, Lq/A, n()V > Context: CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@9 ] by debugging.

yiming-tang-cs · 2018-03-12T21:10:42Z

...unter.streamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/Stream.java

@@ -362,20 +362,20 @@ private IR getEnclosingMethodIR(EclipseProjectAnalysisEngine<InstanceKey> engine
 	}

 	/**
-	 * @return The {@link CGNode} representing the enclosing method of this stream.


This file is from cherry pick and I need your change.

Nope. This should not be part of the change set. No need to cherry-pick. You need to do a merge.

khatchad

Includes changes that are not part of the problem. See comments.

khatchad · 2018-03-12T16:41:03Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+			CallGraph callGraph) {
+		Set<CGNode> deadEntryPoints = new HashSet<CGNode>();
+		Set<String> aliveClass = new HashSet<String>();
+		Set<CGNode> cotersOrStaticInitializerNodes = new HashSet<CGNode>();


It's ctors.

khatchad · 2018-03-12T21:32:35Z

...unter.streamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/Stream.java

@@ -362,20 +362,20 @@ private IR getEnclosingMethodIR(EclipseProjectAnalysisEngine<InstanceKey> engine
 	}

 	/**
-	 * @return The {@link CGNode} representing the enclosing method of this stream.


Nope. This should not be part of the change set. No need to cherry-pick. You need to do a merge.

This reverts commit e90d188.

yiming-tang-cs · 2018-03-24T04:03:47Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+		Set<CGNode> streamNodes = new HashSet<CGNode>();
+		while (streamIterator.hasNext()) {
+			Stream stream = streamIterator.next();
+			streamNodes.addAll(stream.getEnclosingMethodNodes(engine));


I can also catch the NoEnclosingMethodNodeFoundException here to avoid the interruption of a program.

yiming-tang-cs · 2018-03-24T04:15:55Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+			Stream stream = streamIterator.next();
+			try {
+				streamNodes.addAll(stream.getEnclosingMethodNodes(engine));
+			} catch (NoEnclosingMethodNodeFoundException e) {


If the exception is not caught here, the tool cannot produce the correct result for the test case below:

class A { void m() { HashSet h1 = new HashSet(); h1.stream().count(); } @EntryPoint void n() { HashSet h2 = new HashSet(); h2.stream().count(); } }

The program will be interrupted by throwing a NoEnclosingMethodNodeFoundException because there is no CGNode for m() in the call graph.

khatchad · 2018-03-26T15:38:50Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

 			} catch (IOException | CoreException | CancelException e) {
 				LOGGER.log(Level.SEVERE,
 						"Exception encountered while building call graph for: " + project.getElementName() + ".", e);
 				throw new RuntimeException(e);
 			}

+			Collection<CGNode> deadEntryPoints;


Collection<CGNode> deadEntryPoints = new HashSet<>();

khatchad · 2018-03-26T15:39:09Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+			Collection<CGNode> deadEntryPoints;
+			if (!usedEntryPoints.isEmpty())
+				deadEntryPoints = discoverDeadEntryPoints(engine);
+			else


Then, you wouldn't need this else clause.

khatchad · 2018-03-26T15:43:32Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+	 */
+	private Set<CGNode> getStreamCreationNodes(Iterator<Stream> streamIterator,
+			EclipseProjectAnalysisEngine<InstanceKey> engine) {
+		Set<CGNode> streamNodes = new HashSet<CGNode>();


Set<CGNode> streamNodes = new HashSet<>();

khatchad · 2018-03-26T15:45:21Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+	 *            a {@link CallGraph}.
+	 * @return a set of dead entry points.
+	 */
+	private Set<CGNode> getDeadEntryPointNodes(Collection<CGNode> entryPointNodes, Set<CGNode> streamNodes,


Does this have to be an instance method?

khatchad · 2018-03-26T15:48:40Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+		Set<CGNode> ctorsOrStaticInitializerNodes = new HashSet<CGNode>();
+		for (CGNode entryPointNode : entryPointNodes) {
+			// We will process ctors and static initializers later
+			if (Util.isCtors(entryPointNode) || Util.isStaticInitializers(entryPointNode)) {


English: these method names should be singular, e.g., Util.isCtor().

khatchad · 2018-03-26T15:49:34Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+	 * @param callGraph
+	 * @return true: reachable; false: unreachable
+	 */
+	private boolean isReachable(CGNode entryPointNode, Collection<CGNode> streamNodes, CallGraph callGraph) {


Does this need to be an instance method?

khatchad · 2018-03-26T16:11:23Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

 			} catch (IOException | CoreException | CancelException e) {
 				LOGGER.log(Level.SEVERE,
 						"Exception encountered while building call graph for: " + project.getElementName() + ".", e);
 				throw new RuntimeException(e);
 			}

+			Collection<CGNode> deadEntryPoints;
+			if (!usedEntryPoints.isEmpty())
+				deadEntryPoints = discoverDeadEntryPoints(engine);


Can we add an INFO log here for each entry point?

Log one info for each dead entry point (one per line).

khatchad · 2018-03-26T16:21:10Z

Currently, I do not build the call graph again. I have several questions about it.

Where should the tool build the call graph again?

Probably line 243 in https://github.com/ponder-lab/Java-8-Stream-Refactoring/pull/187/files.

The variables, such as ret and projectAnalysisResult should store the information for the first building or the second building?

If you do it on line 243 (above), then usedEntryPoints will reflect the latest build, however, deadEntryPoints will be from the original build.

…tream-Refactoring into master_prune

khatchad · 2018-03-30T14:41:29Z

...reamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/StreamAnalyzer.java

+					// rebuild the callgraph
+					usedEntryPoints = getPrunedEntryPoints(deadEntryPoints, usedEntryPoints);
+					buildCallGraphFromEntryPoints(engine, usedEntryPoints);
+				}


Formatting.

yiming-tang-cs · 2018-03-30T18:11:25Z

I've evaluated the streamql for four times, but the result is not good.

subject	#entrypoint	dead entry point	old version	time (s)
streamql	92	0	no	752.56
streamql	241	18	no	830.43
streamql	259	/	yes	777.425
streamql	92	/	yes	749.496

In my opinion, the reasons may be

This result is gotten occasionally and this can be avoided by evaluating project several times.
The evaluation may cost less time if the call graph is built by less entry points, but the second building call graph may cost too much time.
The number of dead entry points is not so large to reduce enough evaluating time and the reduced time is shorter than difference of evaluating time for each evaluation.

khatchad · 2018-03-30T19:31:54Z

What is old version?

khatchad · 2018-03-30T19:33:21Z

subject #entrypoint dead entry point old version time (s)

streamql 241 18 no 830.43

streamql 259 / yes 777.425

Why are the number of entry points different here?

khatchad · 2018-04-02T14:11:18Z

It sounds to me that if rebuilding the call graph is too expensive, we'll have to find a way to alter the original call graph. But, I would say make sure that this is the operation that is dominating the run time. If, on the other hand, the dominant run time is finding dead entry points, then there's no hope. We should profile this and see where most of the time is being spent, i.e., in the call graph reconstruction or in the finding of the dead entry points.

This reverts commit a3eb676.

This reverts commit 618bdea.

yiming-tang-cs added 2 commits March 8, 2018 15:51

report dead entry points

355cb55

change method name

f8f8dde

yiming-tang-cs requested a review from khatchad as a code owner March 8, 2018 20:57

process coter and static InitializerNodes

36f6bee

ponder-lab deleted a comment from yiming-tang-cs Mar 12, 2018

yiming-tang-cs mentioned this pull request Mar 12, 2018

Need to return a collection of CGNodes for one method #188

Closed

khatchad and others added 6 commits March 12, 2018 14:26

Increase heap size for experiments.

ddb47aa

Fix #188.

e90d188

get a set of entry points

aa1a964

change return

9ba5011

add project result class

d93e047

add test cases

797c675

yiming-tang-cs commented Mar 12, 2018

View reviewed changes

khatchad requested changes Mar 12, 2018

View reviewed changes

khatchad assigned yiming-tang-cs Mar 12, 2018

Revert "Fix #188."

a2d19e8

This reverts commit e90d188.

yiming-tang-cs added 5 commits March 23, 2018 14:08

fix compilcation error

db2d1ec

process catching

01ba9dd

change comments

1517cdd

delete

607c847

change comments

0cf444c

yiming-tang-cs commented Mar 24, 2018

View reviewed changes

noEnclosingMethod

b59706d

yiming-tang-cs commented Mar 24, 2018

View reviewed changes

Call to super() is implicit.

dc57275

ponder-lab deleted a comment from yiming-tang-cs Mar 26, 2018

khatchad requested changes Mar 26, 2018

View reviewed changes

yiming-tang-cs added 4 commits March 29, 2018 00:13

improve and rebuild call graph

0995575

Merge branch 'master_prune' of https://github.com/saledouble/Java-8-S…

a658819

…tream-Refactoring into master_prune

fix no entry point

10c864c

improve logic

d9737d7

khatchad requested changes Mar 30, 2018

View reviewed changes

format

84df4e4

yiming-tang-cs added 7 commits April 15, 2018 16:09

print dead entry points

a3eb676

add printer

96b9fb7

close txt

cc12491

delete unused imported class

618bdea

Revert "print dead entry points"

1c99e3c

This reverts commit a3eb676.

Revert "delete unused imported class"

7fa1e33

This reverts commit 618bdea.

build call graph again

1ccb229

khatchad closed this Oct 4, 2023

Report dead entry points #187

Report dead entry points #187

Uh oh!

Conversation

yiming-tang-cs commented Mar 8, 2018

Uh oh!

yiming-tang-cs commented Mar 8, 2018

Uh oh!

yiming-tang-cs commented Mar 9, 2018

Uh oh!

yiming-tang-cs commented Mar 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yiming-tang-cs commented Mar 9, 2018

Uh oh!

khatchad commented Mar 9, 2018

Uh oh!

khatchad commented Mar 9, 2018

Uh oh!

yiming-tang-cs commented Mar 9, 2018

Uh oh!

khatchad commented Mar 9, 2018 via email

Uh oh!

yiming-tang-cs commented Mar 12, 2018

Uh oh!

yiming-tang-cs commented Mar 12, 2018

Uh oh!

yiming-tang-cs commented Mar 12, 2018

Uh oh!

khatchad commented Mar 12, 2018

Uh oh!

yiming-tang-cs commented Mar 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yiming-tang-cs commented Mar 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

khatchad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

khatchad commented Mar 26, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yiming-tang-cs commented Mar 30, 2018

Uh oh!

khatchad commented Mar 30, 2018

Uh oh!

khatchad commented Mar 30, 2018

Uh oh!

khatchad commented Apr 2, 2018

Uh oh!

yiming-tang-cs commented Mar 9, 2018 •

edited

Loading

yiming-tang-cs commented Mar 12, 2018 •

edited

Loading

yiming-tang-cs commented Mar 12, 2018 •

edited

Loading