[SPARK-17955][SQL] Make DataFrameReader.jdbc call DataFrameReader.format("jdbc").load

[HyukjinKwon]

What changes were proposed in this pull request?

This PR proposes to make DataFrameReader.jdbc call DataFrameReader.format("jdbc").load consistently with other APIs in DataFrameReader/DataFrameWriter and avoid calling sparkSession.baseRelationToDataFrame(..) here and there.

The changes were mostly copied from DataFrameWriter.jdbc() which was recently updated.

-    val params = extraOptions.toMap ++ connectionProperties.asScala.toMap
-    val options = new JDBCOptions(url, table, params)
-    val relation = JDBCRelation(parts, options)(sparkSession)
-    sparkSession.baseRelationToDataFrame(relation)
+    this.extraOptions = this.extraOptions ++ connectionProperties.asScala
+    // explicit url and dbtable should override all
+    this.extraOptions += ("url" -> url, "dbtable" -> table)
+    format("jdbc").load()

How was this patch tested?

Existing tests should cover this.

[SparkQA]

Test build #67011 has finished for PR 15499 at commit 0d6e2d1.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

Merging in master. Thanks.

[gatorsmile]

After this change, we lost the feature for parallel JDBC reading, right?

[gatorsmile]

Let me revert this back.

[HyukjinKwon]

I see. I thought this was all handled in

spark/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcRelationProvider.scala

Lines 29 to 46 in 0c0ad43

	override def createRelation(
	sqlContext: SQLContext,
	parameters: Map[String, String]): BaseRelation = {
	val jdbcOptions = new JDBCOptions(parameters)
	val partitionColumn = jdbcOptions.partitionColumn
	val lowerBound = jdbcOptions.lowerBound
	val upperBound = jdbcOptions.upperBound
	val numPartitions = jdbcOptions.numPartitions
	val partitionInfo = if (partitionColumn == null) {
	null
	} else {
	JDBCPartitioningInfo(
	partitionColumn, lowerBound.toLong, upperBound.toLong, numPartitions.toInt)
	}
	val parts = JDBCRelation.columnPartition(partitionInfo)
	JDBCRelation(parts, jdbcOptions)(sqlContext.sparkSession)
	}

.

Would this be possible to adding those into options rather than reverting this?

[gatorsmile]

We are using a different mechanism for table partitioning. More flexible. Users can do more advanced partitioning (e.g., using multiple columns) here.

[HyukjinKwon]

I see. I am sorry it seems a careless mistake.

[gatorsmile]

nvm