Skip to content

Commit 6b9e8b0

Browse files
maropuankurdave
authored andcommitted
[SPARK-4620] Add unpersist in Graph and GraphImpl
Add an IF to uncache both vertices and edges of Graph/GraphImpl. This IF is useful when iterative graph operations build a new graph in each iteration, and the vertices and edges of previous iterations are no longer needed for following iterations. Author: Takeshi Yamamuro <linguin.m.s@gmail.com> This patch had conflicts when merged, resolved by Committer: Ankur Dave <ankurdave@gmail.com> Closes #3476 from maropu/UnpersistInGraphSpike and squashes the following commits: 77a006a [Takeshi Yamamuro] Add unpersist in Graph and GraphImpl (cherry picked from commit 8817fc7) Signed-off-by: Ankur Dave <ankurdave@gmail.com>
1 parent a4ae7c8 commit 6b9e8b0

File tree

2 files changed

+12
-0
lines changed

2 files changed

+12
-0
lines changed

graphx/src/main/scala/org/apache/spark/graphx/Graph.scala

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,12 @@ abstract class Graph[VD: ClassTag, ED: ClassTag] protected () extends Serializab
104104
*/
105105
def checkpoint(): Unit
106106

107+
/**
108+
* Uncaches both vertices and edges of this graph. This is useful in iterative algorithms that
109+
* build a new graph in each iteration.
110+
*/
111+
def unpersist(blocking: Boolean = true): Graph[VD, ED]
112+
107113
/**
108114
* Uncaches only the vertices of this graph, leaving the edges alone. This is useful in iterative
109115
* algorithms that modify the vertex attributes but reuse the edges. This method can be used to

graphx/src/main/scala/org/apache/spark/graphx/impl/GraphImpl.scala

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,12 @@ class GraphImpl[VD: ClassTag, ED: ClassTag] protected (
7070
replicatedVertexView.edges.checkpoint()
7171
}
7272

73+
override def unpersist(blocking: Boolean = true): Graph[VD, ED] = {
74+
unpersistVertices(blocking)
75+
replicatedVertexView.edges.unpersist(blocking)
76+
this
77+
}
78+
7379
override def unpersistVertices(blocking: Boolean = true): Graph[VD, ED] = {
7480
vertices.unpersist(blocking)
7581
// TODO: unpersist the replicated vertices in `replicatedVertexView` but leave the edges alone

0 commit comments

Comments
 (0)