Skip to content

Commit 4ffaa2e

Browse files
yaooqinnLucaCanali
andcommitted
[SPARK-48289][DOCKER][TEST] Clean up Oracle JDBC tests by skipping redundant SYSTEM password reset
### What changes were proposed in this pull request? This pull request improves the Oracle JDBC tests by skipping the redundant SYSTEM password reset. ### Why are the changes needed? These changes are necessary to clean up the Oracle JDBC tests. This pull request effectively reverts the modifications introduced in [SPARK-46592](https://issues.apache.org/jira/browse/SPARK-46592) and [PR #44594](#44594), which attempted to work around the sporadic occurrence of ORA-65048 and ORA-04021 errors by setting the Oracle parameter DDL_LOCK_TIMEOUT. As discussed in [issue #35](gvenzl/oci-oracle-free#35), setting DDL_LOCK_TIMEOUT did not resolve the issue. The root cause appears to be an Oracle bug or unwanted behavior related to the use of Pluggable Database (PDB) rather than the expected functionality of Oracle itself. Additionally, with [SPARK-48141](https://issues.apache.org/jira/browse/SPARK-48141), we have upgraded the Oracle version used in the tests to Oracle Free 23ai, version 23.4. This upgrade should help address some of the issues observed with the previous Oracle version. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? This patch was tested using the existing test suite, with a particular focus on Oracle JDBC tests. The following steps were executed: ``` export ENABLE_DOCKER_INTEGRATION_TESTS=1 ./build/sbt -Pdocker-integration-tests "docker-integration-tests/testOnly org.apache.spark.sql.jdbc.OracleIntegrationSuite" ``` ### Was this patch authored or co-authored using generative AI tooling? No Closes #46598 from LucaCanali/fixOracleIntegrationTests. Lead-authored-by: Kent Yao <yao@apache.org> Co-authored-by: Luca Canali <luca.canali@cern.ch> Signed-off-by: Kent Yao <yao@apache.org>
1 parent ca35932 commit 4ffaa2e

File tree

4 files changed

+13
-44
lines changed

4 files changed

+13
-44
lines changed

connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala

Lines changed: 0 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,7 @@
1717

1818
package org.apache.spark.sql.jdbc
1919

20-
import java.io.{File, PrintWriter}
21-
22-
import com.github.dockerjava.api.model._
23-
2420
import org.apache.spark.internal.Logging
25-
import org.apache.spark.util.Utils
2621

2722
class OracleDatabaseOnDocker extends DatabaseOnDocker with Logging {
2823
lazy override val imageName =
@@ -38,30 +33,4 @@ class OracleDatabaseOnDocker extends DatabaseOnDocker with Logging {
3833
override def getJdbcUrl(ip: String, port: Int): String = {
3934
s"jdbc:oracle:thin:system/$oracle_password@//$ip:$port/freepdb1"
4035
}
41-
42-
override def beforeContainerStart(
43-
hostConfigBuilder: HostConfig,
44-
containerConfigBuilder: ContainerConfig): Unit = {
45-
try {
46-
val dir = Utils.createTempDir()
47-
val writer = new PrintWriter(new File(dir, "install.sql"))
48-
// SPARK-46592: gvenzl/oracle-free occasionally fails to start with the following error:
49-
// 'ORA-04021: timeout occurred while waiting to lock object', when initializing the
50-
// SYSTEM user. This is due to the fact that the default DDL_LOCK_TIMEOUT is 0, which
51-
// means that the lock will no wait. We set the timeout to 30 seconds to try again.
52-
// TODO: This workaround should be removed once the issue is fixed in the image.
53-
// https://github.com/gvenzl/oci-oracle-free/issues/35
54-
writer.write("ALTER SESSION SET DDL_LOCK_TIMEOUT = 30;\n")
55-
writer.write(s"""ALTER USER SYSTEM IDENTIFIED BY "$oracle_password";""")
56-
writer.close()
57-
val newBind = new Bind(
58-
dir.getAbsolutePath,
59-
new Volume("/docker-entrypoint-initdb.d"),
60-
AccessMode.DEFAULT)
61-
hostConfigBuilder.withBinds(hostConfigBuilder.getBinds :+ newBind: _*)
62-
} catch {
63-
case e: Exception =>
64-
logWarning("Failed to create install.sql file", e)
65-
}
66-
}
6736
}

connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -54,11 +54,11 @@ import org.apache.spark.tags.DockerTest
5454
* A sequence of commands to build the Oracle Database Free container image:
5555
* $ git clone https://github.com/oracle/docker-images.git
5656
* $ cd docker-images/OracleDatabase/SingleInstance/dockerfiles
57-
* $ ./buildContainerImage.sh -v 23.2.0 -f
58-
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.2.0-free
57+
* $ ./buildContainerImage.sh -v 23.4.0 -f
58+
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.4.0-free
5959
*
60-
* This procedure has been validated with Oracle Database Free version 23.2.0,
61-
* and with Oracle Express Edition versions 18.4.0 and 21.3.0
60+
* This procedure has been validated with Oracle Database Free version 23.4.0,
61+
* and with Oracle Express Edition versions 18.4.0 and 21.4.0
6262
*/
6363
@DockerTest
6464
class OracleIntegrationSuite extends DockerJDBCIntegrationSuite with SharedSparkSession {

connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/OracleIntegrationSuite.scala

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -48,12 +48,12 @@ import org.apache.spark.tags.DockerTest
4848
*
4949
* A sequence of commands to build the Oracle Database Free container image:
5050
* $ git clone https://github.com/oracle/docker-images.git
51-
* $ cd docker-images/OracleDatabase/SingleInstance/dockerfiles
52-
* $ ./buildContainerImage.sh -v 23.2.0 -f
53-
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.2.0-free
51+
* $ cd docker-images/OracleDatabase/SingleInstance/dockerfiles0
52+
* $ ./buildContainerImage.sh -v 23.4.0 -f
53+
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.4.0-free
5454
*
55-
* This procedure has been validated with Oracle Database Free version 23.2.0,
56-
* and with Oracle Express Edition versions 18.4.0 and 21.3.0
55+
* This procedure has been validated with Oracle Database Free version 23.4.0,
56+
* and with Oracle Express Edition versions 18.4.0 and 21.4.0
5757
*/
5858
@DockerTest
5959
class OracleIntegrationSuite extends DockerJDBCIntegrationV2Suite with V2JDBCTest {

connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/OracleNamespaceSuite.scala

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,11 +46,11 @@ import org.apache.spark.tags.DockerTest
4646
* A sequence of commands to build the Oracle Database Free container image:
4747
* $ git clone https://github.com/oracle/docker-images.git
4848
* $ cd docker-images/OracleDatabase/SingleInstance/dockerfiles
49-
* $ ./buildContainerImage.sh -v 23.2.0 -f
50-
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.2.0-free
49+
* $ ./buildContainerImage.sh -v 23.4.0 -f
50+
* $ export ORACLE_DOCKER_IMAGE_NAME=oracle/database:23.4.0-free
5151
*
52-
* This procedure has been validated with Oracle Database Free version 23.2.0,
53-
* and with Oracle Express Edition versions 18.4.0 and 21.3.0
52+
* This procedure has been validated with Oracle Database Free version 23.4.0,
53+
* and with Oracle Express Edition versions 18.4.0 and 21.4.0
5454
*/
5555
@DockerTest
5656
class OracleNamespaceSuite extends DockerJDBCIntegrationSuite with V2JDBCNamespaceTest {

0 commit comments

Comments
 (0)