Skip to content

Commit a4cbf26

Browse files
michaelmiorsrowen
authored andcommitted
[SPARK-20637][CORE] Remove mention of old RDD classes from comments
## What changes were proposed in this pull request? A few comments around the code mention RDD classes that do not exist anymore. I'm not sure of the best way to replace these, so I've just removed them here. ## How was this patch tested? Only changes code comments, no testing required Author: Michael Mior <mmior@uwaterloo.ca> Closes #17900 from michaelmior/remove-old-rdds.
1 parent ca4625e commit a4cbf26

File tree

3 files changed

+5
-5
lines changed

3 files changed

+5
-5
lines changed

core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ import org.apache.spark.util._
5858
* set of map output files, and another to read those files after a barrier). In the end, every
5959
* stage will have only shuffle dependencies on other stages, and may compute multiple operations
6060
* inside it. The actual pipelining of these operations happens in the RDD.compute() functions of
61-
* various RDDs (MappedRDD, FilteredRDD, etc).
61+
* various RDDs
6262
*
6363
* In addition to coming up with a DAG of stages, the DAGScheduler also determines the preferred
6464
* locations to run each task on, based on the current cache status, and passes these to the

core/src/test/scala/org/apache/spark/CheckpointSuite.scala

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ trait RDDCheckpointTester { self: SparkFunSuite =>
114114
* RDDs partitions. So even if the parent RDD is checkpointed and its partitions changed,
115115
* the generated RDD will remember the partitions and therefore potentially the whole lineage.
116116
* This function should be called only those RDD whose partitions refer to parent RDD's
117-
* partitions (i.e., do not call it on simple RDD like MappedRDD).
117+
* partitions (i.e., do not call it on simple RDDs).
118118
*
119119
* @param op an operation to run on the RDD
120120
* @param reliableCheckpoint if true, use reliable checkpoints, otherwise use local checkpoints
@@ -388,7 +388,7 @@ class CheckpointSuite extends SparkFunSuite with RDDCheckpointTester with LocalS
388388
// the parent RDD has been checkpointed and parent partitions have been changed.
389389
// Note that this test is very specific to the current implementation of CartesianRDD.
390390
val ones = sc.makeRDD(1 to 100, 10).map(x => x)
391-
checkpoint(ones, reliableCheckpoint) // checkpoint that MappedRDD
391+
checkpoint(ones, reliableCheckpoint)
392392
val cartesian = new CartesianRDD(sc, ones, ones)
393393
val splitBeforeCheckpoint =
394394
serializeDeserialize(cartesian.partitions.head.asInstanceOf[CartesianPartition])
@@ -411,7 +411,7 @@ class CheckpointSuite extends SparkFunSuite with RDDCheckpointTester with LocalS
411411
// Note that this test is very specific to the current implementation of
412412
// CoalescedRDDPartitions.
413413
val ones = sc.makeRDD(1 to 100, 10).map(x => x)
414-
checkpoint(ones, reliableCheckpoint) // checkpoint that MappedRDD
414+
checkpoint(ones, reliableCheckpoint)
415415
val coalesced = new CoalescedRDD(ones, 2)
416416
val splitBeforeCheckpoint =
417417
serializeDeserialize(coalesced.partitions.head.asInstanceOf[CoalescedRDDPartition])

core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -184,7 +184,7 @@ class SparkListenerSuite extends SparkFunSuite with LocalSparkContext with Match
184184
sc.listenerBus.waitUntilEmpty(WAIT_TIMEOUT_MILLIS)
185185
listener.stageInfos.size should be {1}
186186
val stageInfo2 = listener.stageInfos.keys.find(_.stageId == 1).get
187-
stageInfo2.rddInfos.size should be {3} // ParallelCollectionRDD, FilteredRDD, MappedRDD
187+
stageInfo2.rddInfos.size should be {3}
188188
stageInfo2.rddInfos.forall(_.numPartitions == 4) should be {true}
189189
stageInfo2.rddInfos.exists(_.name == "Deux") should be {true}
190190
listener.stageInfos.clear()

0 commit comments

Comments
 (0)