-
Notifications
You must be signed in to change notification settings - Fork 260
Open
Labels
BugSomething isn't workingSomething isn't workingEngine: DatabricksIssues related to DatabricksIssues related to Databricks
Description
The merge
method in the base EngineAdapter
class always appends a WHEN NOT MATCHED THEN INSERT *
after all other match statements - the code in the snippet below always executes.
# This block always executes in the `EngineAdapter.merge` method
match_expressions.append(
exp.When(
matched=False,
source=False,
then=exp.Insert(
this=exp.Tuple(expressions=[exp.column(col) for col in columns_to_types]),
expression=exp.Tuple(
expressions=[
exp.column(col, MERGE_SOURCE_ALIAS) for col in columns_to_types
]
),
),
)
)
This breaks any merges using the WHEN NOT MATCHED BY SOURCE ...
clauses supported by Databricks since the strict order of merge clauses is:
WHEN MATCHED
WHEN NOT MATCHED [BY TARGET]
WHEN NOT MATCHED BY SOURCE
Any deviation from this order results in a syntax error.
I'm happy to contribute to a solution, but some guidance on where is best to make the change would be ideal. I'd guess either defining a custom merge
method for the DatabricksEngineAdapter
or adding a check prior to appending the WHEN NOT MATCHED THEN INSERT *
.
Metadata
Metadata
Assignees
Labels
BugSomething isn't workingSomething isn't workingEngine: DatabricksIssues related to DatabricksIssues related to Databricks