Skip to content
This repository has been archived by the owner on Sep 20, 2023. It is now read-only.

Issue connecting to Athena with multiple profiles, keys, and tokens if not "default" #111

Closed
ErikaJacobs opened this issue Sep 18, 2020 · 2 comments

Comments

@ErikaJacobs
Copy link

ErikaJacobs commented Sep 18, 2020

I'm having difficulty using the profile_name parameter unless the profile's name is "default" in the credentials file. This issue is similar to Issue 51, although it looks like the source code for connection.py has changed since the commit to resolve issue 51 - I wasn't able to find profile_name, token, etc. as arguments in the present source code I looked at. Regardless, those arguments are included in the Python code below.

The credentials file I'm using for AWS changes constantly due to security reasons, and incorporates multiple roles (with none of them named "default"). I'm hoping there's a way to use PyAthenaJBDC to account for changing keys and tokens for multiple profiles when connecting to Athena.

Thank you for your time!

Reproduce Issue:

I have a credentials file that looks like this (~/.aws/credentials):

[user-123]
aws_access_key_id = aaaaaaaa
aws_secret_access_key = bbbbbbbb
aws_session_token = cccccccc

[user-456]
aws_access_key_id = dddddddd
aws_secret_access_key = eeeeeeee
aws_session_token = ffffffff

The code below will not run with the credentials file above:

from pyathenajdbc import connect
from boto3 import Session

aws = Session(profile_name = "user-123")
credentials = aws.get_credentials().get_frozen_credentials()

connect(access_key = credentials.access_key,
secret_key = credentials.secret_key,
token = credentials.token,
profile_name = aws.profile_name,
s3_staging_dir = 's3://BUCKET/',
AwsRegion = aws.region_name
)

However, the code runs if I change the name of one credential to "default":

[default]
aws_access_key_id = aaaaaaaa
aws_secret_access_key = bbbbbbbb
aws_session_token = cccccccc

[user-456]
aws_access_key_id = dddddddd
aws_secret_access_key = eeeeeeee
aws_session_token = ffffffff

Then change the Python code to have profile_name as "default":

from pyathenajdbc import connect
from boto3 import Session

aws = Session(profile_name = "default")
credentials = aws.get_credentials().get_frozen_credentials()

connect(access_key = credentials.access_key,
secret_key = credentials.secret_key,
token = credentials.token,
profile_name = aws.profile_name,
s3_staging_dir = 's3://BUCKET/',
AwsRegion = aws.region_name
)

@laughingman7743
Copy link
Owner

laughingman7743 commented Sep 19, 2020

Starting in version 2.1.0, JDBC driver options are now specified as arguments to the connect method.
https://github.com/laughingman7743/PyAthenaJDBC/releases/tag/v2.1.0

connect(Profile= aws.profile_name,
S3OutputLocation = 's3://BUCKET/',
AwsRegion = aws.region_name
)

See the documentation for JDBC driver options.
https://s3.amazonaws.com/athena-downloads/drivers/JDBC/SimbaAthenaJDBC_2.0.13/docs/Simba+Athena+JDBC+Driver+Install+and+Configuration+Guide.pdf

It is recommended to use the Boto3 version rather than the JDBC version.
https://github.com/laughingman7743/PyAthena

@laughingman7743
Copy link
Owner

#112 (comment) 🙇‍♂️

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants