[KYUUBI #5367] Spark crashes with ClassCastException when resolving a join of masked tables

[pilietis93]

🔍 Description

Issue References 🔗

This pull request fixes #5092

Spark crashes with ClassCastException when resolving a join of masked tables

Describe Your Solution 🔧

As per @sauliusvl comment in this unfinished PR Stage0Marker column ids are not updated on the right side of the join which leads to Stage1Marker being incorrectly applied which in turn causes

java.lang.ClassCastException: org.apache.kyuubi.plugin.spark.authz.ranger.datamasking.DataMaskingStage1Marker cannot be cast to org.apache.spark.sql.catalyst.plans.logical.Join

So the solution was to copy the scan plan together with the child plan in withNewChildInternal method so that column ids are aligned.

Types of changes 🔖

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Behavior Without This Pull Request ⚰️

Self join of tables containing masked columns using joinExprs arg fails with ClassCastException.

Behavior With This Pull Request 🎉

Self join of tables containing masked columns using joinExprs arg succeeds.

Related Unit Tests

DataMaskingTestBase

---

Checklist 📝

• This patch was not authored or co-authored using Generative Tooling

Be nice. Be informative.

[pan3793]

The change makes sense to me, thanks for contribution!

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 58.45%. Comparing base (01187ed) to head (9906a53).
Report is 1 commits behind head on master.

Files	Patch %	Lines
...thz/rule/datamasking/DataMaskingStage0Marker.scala	50.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             master    #6012   +/-   ##
=========================================
  Coverage     58.44%   58.45%           
  Complexity       24       24           
=========================================
  Files           653      653           
  Lines         39865    39883   +18     
  Branches       5481     5482    +1     
=========================================
+ Hits          23299    23312   +13     
- Misses        14073    14077    +4     
- Partials       2493     2494    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pan3793]

seems the test failed in Spark 3.1 and 3.5