You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-16911] Fix the links in the programming guide
## What changes were proposed in this pull request?
Fix the broken links in the programming guide of the Graphx Migration and understanding closures
## How was this patch tested?
By running the test cases and checking the links.
Author: Shivansh <shiv4nsh@gmail.com>
Closes#14503 from shiv4nsh/SPARK-16911.
(cherry picked from commit 6c1ecb1)
Signed-off-by: Sean Owen <sowen@cloudera.com>
[aggregateMessages](#aggregateMessages)) as well as an optimized variant of the [Pregel](#pregel) API. In addition, GraphX includes a growing collection of graph [algorithms](#graph_algorithms) and
68
68
[builders](#graph_builders) to simplify graph analytics tasks.
69
69
70
-
71
-
## Migrating from Spark 1.1
72
-
73
-
GraphX in Spark 1.2 contains a few user facing API changes:
74
-
75
-
1. To improve performance we have introduced a new version of
76
-
[`mapReduceTriplets`][Graph.mapReduceTriplets] called
77
-
[`aggregateMessages`][Graph.aggregateMessages] which takes the messages previously returned from
78
-
[`mapReduceTriplets`][Graph.mapReduceTriplets] through a callback ([`EdgeContext`][EdgeContext])
79
-
rather than by return value.
80
-
We are deprecating [`mapReduceTriplets`][Graph.mapReduceTriplets] and encourage users to consult
81
-
the [transition guide](#mrTripletsTransition).
82
-
83
-
2. In Spark 1.0 and 1.1, the type signature of [`EdgeRDD`][EdgeRDD] switched from
84
-
`EdgeRDD[ED]` to `EdgeRDD[ED, VD]` to enable some caching optimizations. We have since discovered
85
-
a more elegant solution and have restored the type signature to the more natural `EdgeRDD[ED]` type.
86
-
87
70
# Getting Started
88
71
89
72
To get started you first need to import Spark and GraphX into your project, as follows:
Copy file name to clipboardExpand all lines: docs/programming-guide.md
+1-44Lines changed: 1 addition & 44 deletions
Original file line number
Diff line number
Diff line change
@@ -1097,7 +1097,7 @@ for details.
1097
1097
<tr>
1098
1098
<td> <b>foreach</b>(<i>func</i>) </td>
1099
1099
<td> Run a function <i>func</i> on each element of the dataset. This is usually done for side effects such as updating an <ahref="#accumulators">Accumulator</a> or interacting with external storage systems.
1100
-
<br /><b>Note</b>: modifying variables other than Accumulators outside of the <code>foreach()</code> may result in undefined behavior. See <ahref="#ClosuresLink">Understanding closures </a> for more details.</td>
1100
+
<br /><b>Note</b>: modifying variables other than Accumulators outside of the <code>foreach()</code> may result in undefined behavior. See <ahref="#understanding-closures-a-nameclosureslinka">Understanding closures </a> for more details.</td>
1101
1101
</tr>
1102
1102
</table>
1103
1103
@@ -1544,49 +1544,6 @@ and then call `SparkContext.stop()` to tear it down.
1544
1544
Make sure you stop the context within a `finally` block or the test framework's `tearDown` method,
1545
1545
as Spark does not support two contexts running concurrently in the same program.
1546
1546
1547
-
# Migrating from pre-1.0 Versions of Spark
1548
-
1549
-
<divclass="codetabs">
1550
-
1551
-
<divdata-lang="scala"markdown="1">
1552
-
1553
-
Spark 1.0 freezes the API of Spark Core for the 1.X series, in that any API available today that is
1554
-
not marked "experimental" or "developer API" will be supported in future versions.
1555
-
The only change for Scala users is that the grouping operations, e.g. `groupByKey`, `cogroup` and `join`,
1556
-
have changed from returning `(Key, Seq[Value])` pairs to `(Key, Iterable[Value])`.
1557
-
1558
-
</div>
1559
-
1560
-
<divdata-lang="java"markdown="1">
1561
-
1562
-
Spark 1.0 freezes the API of Spark Core for the 1.X series, in that any API available today that is
1563
-
not marked "experimental" or "developer API" will be supported in future versions.
1564
-
Several changes were made to the Java API:
1565
-
1566
-
* The Function classes in `org.apache.spark.api.java.function` became interfaces in 1.0, meaning that old
1567
-
code that `extends Function` should `implement Function` instead.
1568
-
* New variants of the `map` transformations, like `mapToPair` and `mapToDouble`, were added to create RDDs
1569
-
of special data types.
1570
-
* Grouping operations like `groupByKey`, `cogroup` and `join` have changed from returning
1571
-
`(Key, List<Value>)` pairs to `(Key, Iterable<Value>)`.
1572
-
1573
-
</div>
1574
-
1575
-
<divdata-lang="python"markdown="1">
1576
-
1577
-
Spark 1.0 freezes the API of Spark Core for the 1.X series, in that any API available today that is
1578
-
not marked "experimental" or "developer API" will be supported in future versions.
1579
-
The only change for Python users is that the grouping operations, e.g. `groupByKey`, `cogroup` and `join`,
1580
-
have changed from returning (key, list of values) pairs to (key, iterable of values).
1581
-
1582
-
</div>
1583
-
1584
-
</div>
1585
-
1586
-
Migration guides are also available for [Spark Streaming](streaming-programming-guide.html#migration-guide-from-091-or-below-to-1x),
1587
-
[MLlib](ml-guide.html#migration-guide) and [GraphX](graphx-programming-guide.html#migrating-from-spark-091).
1588
-
1589
-
1590
1547
# Where to Go from Here
1591
1548
1592
1549
You can see some [example Spark programs](http://spark.apache.org/examples.html) on the Spark website.
0 commit comments