-
Notifications
You must be signed in to change notification settings - Fork 1k
RANGER-2128: Implementation of Ranger Spark SQL plugin #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@boscodurai ship it? |
keen to see this in action! do you have any jar and setup notes for this? |
@yaooqinn can u create apache review? |
@tooptoop4 review request created. |
Hi! This work is very interesting! |
@yaooqinn Can you fix the conflicts? |
@yaooqinn any thoughts on resolving the conflicts? |
What does this PR do/accomplish that isn't already possible with the existing Hive support? We're currently running Spark Thriftserver (3.2.x) with the kyuubi plugin against Ranger where in Ranger we've defined the service as a Hive service and everything with regards to authentication and authorization seems to be working as expected. The only thing that I've observed that doesn't work is the auto-complete when creating policies via the Ranger UI, I assume this is a slight dialect difference in the response from the Spark Thriftserver vs a real HiveServer2 since the query being run by Ranger ( |
Thanks @simonvanderveldt. I think than we can include Kyuubi plugin in Ranger for Spark. |
+1 |
will this work with spark-submit cluster mode without passing keytab? |
I am going to close this in favor of the kyuubi spark authz plugin |
pre-work
Basic concepts and introductions can be found in spark-authorizer's documentations.
additionals
apache/spark#17724 exposed a new experimental develop api
SparkSessionExetensions
, which is able to add user supplied extensions toSparkSession
object during instantiation via program api or the spark property namedspark.sql.extensions
.This PR uses
spark.sql.extensions
and other necessary ranger-hive-plugin settings to enable Ranger security support for Spark SQL with hive as external catalog.