Skip to content

[SPARK-46608][SQL] Restore backward compatibility of JdbcDialect.classifyException #44449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 16 commits into from

Conversation

MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Dec 21, 2023

What changes were proposed in this pull request?

In the PR, I propose to restore classifyException() of JdbcDialect before the commit 14a933b, and extends classifyException() with the error class parameter by description:

  def classifyException(
      e: Throwable,
      errorClass: String,
      messageParameters: Map[String, String],
      description: String): AnalysisException

The description parameter has the same meaning as message in the old version of classifyException() which is deprecated.

Also old implementation of classifyException() has been restored in JDBC dialects: MySQL, PostgreSQL and so on.

Why are the changes needed?

To restore compatibility with existing JDBC dialects.

Does this PR introduce any user-facing change?

No, this PR restores the behaviour prior #44358.

How was this patch tested?

By running the affected test suite:

$ build/sbt "core/testOnly *SparkThrowableSuite"

and modified test suite:

$ build/sbt "test:testOnly *JDBCV2Suite"
$ build/sbt "test:testOnly *JDBCTableCatalogSuite"

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the SQL label Dec 21, 2023
@github-actions github-actions bot added the DOCS label Dec 21, 2023
@MaxGekk MaxGekk changed the title [WIP][SQL] Restore backward compatibility of JdbcDialect.classifyException [WIP][SPARK-46608][SQL] Restore backward compatibility of JdbcDialect.classifyException Jan 5, 2024
@MaxGekk MaxGekk changed the title [WIP][SPARK-46608][SQL] Restore backward compatibility of JdbcDialect.classifyException [SPARK-46608][SQL] Restore backward compatibility of JdbcDialect.classifyException Jan 6, 2024
@MaxGekk MaxGekk marked this pull request as ready for review January 6, 2024 16:56
@MaxGekk MaxGekk requested a review from cloud-fan January 6, 2024 16:57
new AnalysisException(errorClass, messageParameters, cause = Some(e))
messageParameters: Map[String, String],
description: String): AnalysisException = {
classifyException(description, e)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of passing description around, shall we generate the description here using the error class name and the tableName in the parameters?

Copy link
Member Author

@MaxGekk MaxGekk Jan 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be not fully compatible with existing JDBC dialects because values in messageParameters has been preprocessed already like quoting, and a JDBC dialect could use regexp for parsing parameters from message/description. Theoretically, it can break the regexps.

And if we pass the quoting values to an JDBC dialect, it can throw some Spark exception which performs quoting inside like PostgresDialect throws NonEmptyNamespaceException. The last one does quoting inside its constructor. Apparently, we will get double quoting values which is a bug already.

@MaxGekk
Copy link
Member Author

MaxGekk commented Jan 8, 2024

Merging to master. Thank you, @cloud-fan for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants