Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AWS] Add identity info when check fails #2456

Merged
merged 7 commits into from
Aug 24, 2023
Merged

Conversation

Michaelvll
Copy link
Collaborator

This adds more observability for #2451

Tested (run the relevant ones):

  • Code formatting: bash format.sh
  • Any manual or new tests for this PR (please specify below)
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

sky/clouds/service_catalog/data_fetchers/fetch_aws.py Outdated Show resolved Hide resolved
'likely due to permission issues, please check the minimal '
'permission required for AWS: https://skypilot.readthedocs.io/en/latest/cloud-setup/cloud-permissions/aws.html' # pylint: disable=
'Failed to fetch the availability zones for the account '
f'{identity}. It is likely due to permission issues, please '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
f'{identity}. It is likely due to permission issues, please '
f'{identity!r}. It is likely due to permission issues, please '

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a list of strings. We probably don't want to add !r

sky/clouds/service_catalog/data_fetchers/fetch_aws.py Outdated Show resolved Hide resolved
'Please ensure that the `ec2:DescribeAvailabilityZones` '
'action is enabled for your AWS account in IAM. '
'Ref: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeAvailabilityZones.html' # pylint: disable=line-too-long
'Ref: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeAvailabilityZones.html.\n' # pylint: disable=line-too-long

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more detail on this raise would be good - currently just recieve the error: "Details: [botocore.exceptions.ClientError] An error occurred (UnauthorizedOperation) when calling the DescribeAvailabilityZones operation: You are not authorized to perform this operation."

despite: aws ec2 describe-availability-zones working fine.

Can you log the account at this point also?

Copy link
Collaborator Author

@Michaelvll Michaelvll Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment @TRT-BradleyB! The upper-level caller in the aws.py should log the identity information as modified above. Do you see that message when running sky check?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did think that was the intention, this is the extent of the message I recieve:

RuntimeError: Failed to retrieve availability zones. Please ensure that the `ec2:DescribeAvailabilityZones` action is enabled for your AWS account in IAM. Ref: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeAvailabilityZones.html.
Details: [botocore.exceptions.ClientError] An error occurred (UnauthorizedOperation) when calling the DescribeAvailabilityZones operation: You are not authorized to perform this operation.

Copy link
Collaborator Author

@Michaelvll Michaelvll Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this is weird. I did see the following messages when running sky check, where the first line of the Reason shows the account info:

$ sky check                                                     
Checking credentials to enable clouds for SkyPilot.
  Checking AWS...I 08-24 10:09:34 aws_catalog.py:79] Fetching availability zones mapping for AWS...
  AWS: disabled          
    Reason: Failed to fetch the availability zones for the account ['my-account', 'my-account']. It is likely due to permission issues, please check the minimal permission required for AWS: https://skypilot.readthedocs.io/en/latest/cloud-setup/cloud-permissions/aws.html
    Details: [builtins.RuntimeError] Failed to retrieve availability zones. Please ensure that the `ec2:DescribeAvailabilityZones` action is enabled for your AWS account in IAM. Ref: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeAvailabilityZones.html. Details: [botocore.exceptions.ClientError] ...

Just to confirm, is the error you shown from sky check?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah - no that's my bad I was running sky launch - check indeed has it.

Copy link
Collaborator Author

@Michaelvll Michaelvll Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming! Merging this for now, and we can continue the discussion in #2451 for the availability zones permission issue. ; )

@Michaelvll Michaelvll merged commit c91a11c into master Aug 24, 2023
17 checks passed
@Michaelvll Michaelvll deleted the add-identity-in-check branch August 24, 2023 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants