-
Notifications
You must be signed in to change notification settings - Fork 8.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HDFS-13248: Namenode needs to use the actual client IP when going through RBF proxy. #4081
Conversation
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
Outdated
Show resolved
Hide resolved
...ject/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
Show resolved
Hide resolved
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
.../hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeRpcServer.java
Outdated
Show resolved
Hide resolved
.../hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeRpcServer.java
Outdated
Show resolved
Hide resolved
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
Show resolved
Hide resolved
...ject/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
Show resolved
Hide resolved
.../hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeRpcServer.java
Show resolved
Hide resolved
.../hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeRpcServer.java
Show resolved
Hide resolved
.../hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeRpcServer.java
Outdated
Show resolved
Hide resolved
...ject/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
Outdated
Show resolved
Hide resolved
.../hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeRpcServer.java
Show resolved
Hide resolved
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
public void testNamenodeRpcClientIpProxy() throws IOException { | ||
// Because of the randomness of the NN assigning DN, we run multiple | ||
// trials. 1/3^20=3e-10, so that should be good enough. | ||
final int ITERATIONS_TO_USE = 20; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
variable name fetches checkstyle warning
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4081/5/artifact/out/results-checkstyle-root.txt
💔 -1 overall
This message was automatically generated. |
+1, TestRouterDistCpProcedure isn't related, we should track that separately |
…ough the RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures the list of users than can set their client ip address using the client context. Fixes apache#4081
💔 -1 overall
This message was automatically generated. |
…ough the RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures the list of users than can set their client ip address using the client context. Fixes #4081
…ough the RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures the list of users than can set their client ip address using the client context. Fixes apache#4081
…ough the RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures the list of users than can set their client ip address using the client context. Fixes apache#4081
…ough the RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures the list of users than can set their client ip address using the client context. Fixes apache#4081
…ough the RBF proxy. There is a new configuration knob dfs.namenode.ip-proxy-users that configures the list of users than can set their client ip address using the client context. Fixes apache#4081 RB=3356942 G=superfriends-reviewers R=sdzinama A=sdzinama
The NN makes decisions based on the client machine that control the locality of data access.
Currently that is done by finding the ip address using the rpc connection, however in the RBF
configuration, that will always be one of the router's ip address.
We'd added the client's ip to the caller context in the router, so now the NN has the information.
This patch makes the NN use the caller context information.
From a security point of view, this patch adds a new configuration knob (dfs.namenode.ip-proxy-users) on the NN
that defines the list of users that can set their client ip address. Sites should add "hdfs" (or the account that
runs the routers) to "dfs.namenode.ip-proxy-users" on the NN to enable this feature.
Note that the audit log does NOT currently use this information, so the client ip in the audit log will be the RBF proxy.
Sites should turn on caller context logging so that the client ip addresses are captured.
Description of PR
How was this patch tested?
For code changes:
LICENSE
,LICENSE-binary
,NOTICE-binary
files?