Skip to content

Add kerberos keytab authorization #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

jo8937
Copy link

@jo8937 jo8937 commented Oct 6, 2022

Thank you for your great work. embulk-output-hdfs is very cool plugin.
I want to add some feature - kerberos keytab auth.
use like this.

  • command line auth
kinit -kt user.keytab user
  • embulk yaml file
out:
  ....
  type: hdfs
  path_prefix: ./testdata
  file_ext: tsv
  config_files:
    - ./hadoop-conf/core-site.xml
    - ./hadoop-conf/hdfs-site.xml
  doas: user
  keytab_config:
    keytab_path: ./env/user.keytab
    keytab_principal: user@CLOUDERA.COM
    krb5_config_path: ./env/krb5.conf
  config:
    fs.defaultFS: 'webhdfs://my-hadoop-server:9870'
    fs.webhdfs.impl: 'org.apache.hadoop.hdfs.web.WebHdfsFileSystem'
    hadoop.security.authentication: 'kerberos'
    hadoop.security.authorization: 'true'
    dfs.client.use.datanode.hostname: 'true'
    dfs.namenode.kerberos.principal.pattern: 'hdfs/*@CLOUDERA.COM'    
   ....

This pull request's implementation is working fine in my environment with "webhdfs"

@jo8937 jo8937 closed this Oct 6, 2022
@jo8937
Copy link
Author

jo8937 commented Oct 6, 2022

Sorry for invalid PR.

@jo8937 jo8937 reopened this Oct 6, 2022
@jo8937
Copy link
Author

jo8937 commented Oct 6, 2022

The keytab feature has some errors with hdfs scheme, but It works fine with webhdfs scheme
maybe it is useful for some user

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant