How to type-annotate ServiceErrors

[michaelrommel]

I am trying to handle an UnauthorizedException gracefully with a user friendly error message.
The code path shown below is taken, when a call to S3 fails due to a missing or invalid token. The default error message looks like this (structured a bit for more clarity):

DispatchFailure(
	DispatchFailure { 
		source: ConnectorError { 
			kind: Other(None), 
			source: ProviderError(
				ProviderError { 
					source: ProviderError(
						ProviderError { 
							source: ServiceError(
								ServiceError { 
									source: UnauthorizedException(
										UnauthorizedException { 
											message: Some("Session token not found or invalid"), 
											meta: ErrorMetadata { 
												code: Some("UnauthorizedException"), 
												message: Some("Session token not found or invalid"), 
												extras: Some({"aws_request_id": "1b6d7476-f5ec-4a16-9890-7684ccee7d01"})
											} 
										}
									), 
									raw: Response {
										status: StatusCode(401), 
										headers: Headers {
											headers: {
												"date": HeaderValue { _private: H0("Thu, 04 Jul 2024 07:41:21 GMT") }, 
												"content-type": HeaderValue { _private: H0("application/json") }, 
												"content-length": HeaderValue { _private: H0("114") }, 
												"access-control-expose-headers": HeaderValue { _private: H0("RequestId") }, 
												"access-control-expose-headers": HeaderValue { _private: H0("x-amzn-RequestId") }, 
												"requestid": HeaderValue { _private: H0("1b6d7476-f5ec-4a16-9890-7684ccee7d01") }, 
												"server": HeaderValue { _private: H0("AWS SSO") }, 
												"x-amzn-requestid": HeaderValue { _private: H0("1b6d7476-f5ec-4a16-9890-7684ccee7d01") }
											} 
										}, 
										body: SdkBody {
											inner: Once(
												Some(
													b"{
														\"message\":\"Session token not found or invalid\",
														\"__type\":\"com.amazonaws.switchboard.portal#UnauthorizedException\"}"
													)
												), 
											retryable: true 
										}, 
										extensions: Extensions {
											extensions_02x: Extensions, 
											extensions_1x: Extensions 
										}
									} 
								}
							) 
						}
					) 
				}
			), 
			connection: Unknown 
		} 
	}
)

The code where I am trying to handle those nested errors looks at the moment like this:

            Err(SdkError::DispatchFailure(df)) => {
                println!("DispatchFailure {:?}", df);
                match df.as_connector_error() {
                    Some(err) => {
                        println!("ConnectorError is {:?}", err);
                        let ce_source: Option<&CredentialsError> =
                            err.source().unwrap().downcast_ref();
                        match ce_source {
                            Some(err) => {
                                println!("Error {:?}", err);
                                let pe_source: Option<&CredentialsError> =
                                    err.source().unwrap().downcast_ref();
                                match pe_source {
                                    Some(err) => {
                                        println!("Error {:?}", err);
                                        let pe_source2: Option<&ServiceError> =
                                            err.source().unwrap().downcast_ref();
                                        match pe_source2 {
                                            Some(err) => {
                                                println!("Service Error: {:?}", err);
                                            }
                                            _ => todo!(),
                                        }
                                    }
                                    _ => todo!(),
                                }
                            }
                            _ => todo!(),
                        }
                    }
                    None => {
                        panic!("No Connector Error");
                    }
                }
            }

This fails, because the type annotation for the ServiceError expects two types <E, R> an I cannot for the life of me figure out, where in all the aws sdks the types for an "UnauthorizedException" or a "Response" are defined and what I can put there instead of E and R.

Also, overall I am quite puzzled, that it is such a hassle to get to the underlying root cause of an error. The overall code is like 200 lines and already 40 of them are just to get the first error handled, not to mention all the open todo!s.

I would like to say, that it is my lack of expertise with this sdk, but since I have now searched for a proper error handling guidance for 8 hours and found so many struggling with this, that I tend to think that there is something severely lacking, be it documentation, examples or code...

Is there someone out there, who has implemented error handling in a Rust idiomatic way, that is concise and clear to understand?

Thank you so much in advance!

[michaelrommel]

So now I found the types as:

let se: Option<&ServiceError<aws_sdk_schemas::types::error::UnauthorizedException, aws_smithy_runtime_api::client::orchestrator::HttpResponse>> =
        err.source().unwrap().downcast_ref();

which does not compile either, because of:

 1  error[E0277]: the trait bound `aws_smithy_runtime_api::client::result::ServiceError<aws_sdk_schemas::types::error::UnauthorizedExceptio
 n, aws_smithy_runtime_api::http::Response>: std::error::Error` is not satisfied
    --> src/bin/fetch-objects-tracking.rs:122:67
     |
 122 | ...                   err.source().unwrap().downcast_ref();
     |                                             ^^^^^^^^^^^^ the trait `std::error::Error` is not implemented for `ServiceError<Unauthor
 izedException, Response>`
     |
 note: required by a bound in `<(dyn std::error::Error + 'static)>::downcast_ref`
    --> /home/rommel/.local/share/mise/installs/rust/1.78.0/toolchains/1.78.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/sr
 c/error.rs:218:28
     |
 218 |     pub fn downcast_ref<T: Error + 'static>(&self) -> Option<&T> {
     |                            ^^^^^ required by this bound in `<dyn Error>::downcast_ref`

So nothing really works. Not the .into_service_error way, the destructuring way, the downcast way...

I give up. Error handling is unusable without better documentation. Enough time wasted.

[ysaito1001]

We apologize for having you spend long hours trying to obtain what you need.

Is the end goal of the error handling above accessing the underlying error message from S3? If so, you can try using DisplayErrorContext as suggested here.

[michaelrommel]

Thank you, @ysaito1001 and @Velfi, for replying and offering a solution, it is highly appreciated.

It is my fault that I have not described better, what I am trying to achieve, so let me try again with some concrete minimalistic code examples.

If I start from one of the basic examples from the SDK and I set up my AWS profiles and sso tokens correctly, then I get correctly the bucket name back:

> aws sso login --profile test
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.us-east-1.amazonaws.com/

Then enter the code:

XXXX-XXXX
Successfully logged into Start URL: https://shsconsole.awsapps.com/start
> cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/debug/test`
xxxxx-xxxxxxxxxxxx-xxxxx-xxxxxxxx-123456789012

Code:

#![allow(clippy::result_large_err)]

use aws_config::{meta::region::RegionProviderChain, BehaviorVersion};
use aws_sdk_s3::Client;

async fn show_buckets(client: &Client) -> Result<(), aws_sdk_s3::Error> {
    let resp = client.list_buckets().send().await?;
    let buckets = resp.buckets();

    for bucket in buckets {
        println!("{}", bucket.name().unwrap_or_default());
    }

    Ok(())
}

#[tokio::main]
async fn main() -> Result<(), aws_sdk_s3::Error> {
    let profile_name = "test";

    let shared_config = aws_config::defaults(BehaviorVersion::v2024_03_28())
        .credentials_provider(
            aws_config::profile::ProfileFileCredentialsProvider::builder()
                .profile_name(profile_name)
                .build(),
        )
        .token_provider(
            aws_config::profile::ProfileFileTokenProvider::builder()
                .profile_name(profile_name)
                .build(),
        )
        .region(
            RegionProviderChain::first_try(
                aws_config::profile::ProfileFileRegionProvider::builder()
                    .profile_name(profile_name)
                    .build(),
            )
            .region()
            .await
            .unwrap(),
        )
        .load()
        .await;

    let client = Client::new(&shared_config);

    show_buckets(&client).await
}

If I logout again, I get this:

> aws sso logout --profile test

> cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.32s
     Running `target/debug/test`
Error: Unhandled(Unhandled { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: ProviderError(ProviderError { source: ProviderError(ProviderError { source: ProviderError(ProviderError { source: FailedToLoadToken { source: IoError { what: "read", path: "/home/rommel/.aws/sso/cache/2c7dab8fb022b6e5f2932276796109768570951f.json", source: Os { code: 2, kind: NotFound, message: "No such file or directory" } } } }) }) }), connection: Unknown } }), meta: ErrorMetadata { code: None, message: None, extras: None } })

This error message is different, when the token is still there, but expired:

Error: Unhandled(Unhandled { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: ProviderError(ProviderError { source: ProviderError(ProviderError { source: ServiceError(ServiceError { source: UnauthorizedException(UnauthorizedException { message: Some("Session token not found or invalid"), meta: ErrorMetadata { code: Some("UnauthorizedException"), message: Some("Session token not found or invalid"), extras: Some({"aws_request_id": "294e0b5e-5cbb-47fd-9808-1c11a8721217"}) } }), raw: Response { status: StatusCode(401), headers: Headers { headers: {"date": HeaderValue { _private: H0("Mon, 08 Jul 2024 07:12:51 GMT") }, "content-type": HeaderValue { _private: H0("application/json") }, "content-length": HeaderValue { _private: H0("114") }, "access-control-expose-headers": HeaderValue { _private: H0("RequestId") }, "access-control-expose-headers": HeaderValue { _private: H0("x-amzn-RequestId") }, "requestid": HeaderValue { _private: H0("294e0b5e-5cbb-47fd-9808-1c11a8721217") }, "server": HeaderValue { _private: H0("AWS SSO") }, "x-amzn-requestid": HeaderValue { _private: H0("294e0b5e-5cbb-47fd-9808-1c11a8721217") }} }, body: SdkBody { inner: Once(Some(b"{\"message\":\"Session token not found or invalid\",\"__type\":\"com.amazonaws.switchboard.portal#UnauthorizedException\"}")), retryable: true }, extensions: Extensions { extensions_02x: Extensions, extensions_1x: Extensions } } }) }) }), connection: Unknown } }), meta: ErrorMetadata { code: None, message: None, extras: None } })

Or if the region is misconfigured:

Error: Unhandled(Unhandled { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Io, source: hyper::Error(Connect, ConnectError("dns error", Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" })), connection: Unknown } }), meta: ErrorMetadata { code: None, message: None, extras: None } })

My goal was to distinguish between different error messages and give the user a more helpful information on how to overcome the respective problem. Here in both cases, a re-login is necessary, but there may be other kinds of errors, that require a different fix, like to correct the region.

If I now change the code to use the DisplayErrorContext() method,

async fn show_buckets(client: &Client) -> Result<(), aws_sdk_s3::Error> {
    let response = client.list_buckets().send().await;
    match response {
        Ok(resp) => {
            let buckets = resp.buckets();
            for bucket in buckets {
                println!("{}", bucket.name().unwrap_or_default());
            }
        }
        Err(err) => {
            panic!("Error x occurred: {:?}", DisplayErrorContext(err))
        }
    }

    Ok(())
}

this would only be a slight change:

thread 'main' panicked at src/main.rs:16:13:
Error x occurred: DisplayErrorContext(DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: ProviderError(ProviderError { source: ProviderError(ProviderError { source: ServiceError(ServiceError { source: UnauthorizedException(UnauthorizedException { message: Some("Session token not found or invalid"), meta: ErrorMetadata { code: Some("UnauthorizedException"), message: Some("Session token not found or invalid"), extras: Some({"aws_request_id": "365e5a1a-688c-4ad0-8360-0d0a54a82114"}) } }), raw: Response { status: StatusCode(401), headers: Headers { headers: {"date": HeaderValue { _private: H0("Mon, 08 Jul 2024 07:27:27 GMT") }, "content-type": HeaderValue { _private: H0("application/json") }, "content-length": HeaderValue { _private: H0("114") }, "access-control-expose-headers": HeaderValue { _private: H0("RequestId") }, "access-control-expose-headers": HeaderValue { _private: H0("x-amzn-RequestId") }, "requestid": HeaderValue { _private: H0("365e5a1a-688c-4ad0-8360-0d0a54a82114") }, "server": HeaderValue { _private: H0("AWS SSO") }, "x-amzn-requestid": HeaderValue { _private: H0("365e5a1a-688c-4ad0-8360-0d0a54a82114") }} }, body: SdkBody { inner: Once(Some(b"{\"message\":\"Session token not found or invalid\",\"__type\":\"com.amazonaws.switchboard.portal#UnauthorizedException\"}")), retryable: true }, extensions: Extensions { extensions_02x: Extensions, extensions_1x: Extensions } } }) }) }), connection: Unknown } }))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

So the shown message changes from Error: Unhandled(Unhandled { source: to Error x occurred: DisplayErrorContext(. What I wanted to achive is to get access to the underlying reason, so that I can programmatically tell the user what to do and only panic in case of an unkown reason that the code is not prepared to handle.

So I want to get at the ServiceError { source: UnauthorizedException(UnauthorizedException { message: Some("Session token not found or invalid") portion of the error and all the functions that I found could not do that: The .into_service_error function on the top level error just wawaps the whole string in just the same Unhandled(Unhandled...) wrapper. If I try to extract the levels one by one by downcasting the error like I showed in my very first code snippet, then I get down to the ServiceError, but I am unable to downcast the last level to a ServiceError because of the missing trait implementation:

the trait `std::error::Error` is not implemented for `ServiceError<UnauthorizedException, Response>`

And the .into_service_error() method also does not work, because it is not a function of the plain Error object that I have at this level.

Therefore my question, what am I doing wrong. I am a novice in rust with a few months experience, so I totally blame this on my lack of knowledge and so I ask for guidance what the correct way of achieving my goal is, or of it is nothing I should concern myself with and let the errors speak for themselves.

Thank you for taking the time to read all of this and if you can help out with a small code example, that would be very much appreciated.

Michael.

[ysaito1001]

Thank you for taking time to clarify your use case.

What I wanted to achive is to get access to the underlying reason, so that I can programmatically tell the user what to do

Sounds like you wish to write an error handling code with a match expression that has arms based on the underlying error reasons?

One thing to note is that the errors you shared with us above indicate they are wrapped in DispatchError, which is an informative error. You can take a look at this RFC for more details, but the gist is that there are two broad categories of errors in the SDK, actionable and informative. It makes sense to write a match expression on the former but makes less sense to do so with the latter, because the latter only carries the opaque error type and the most meaningful thing you can do with it is to print it.

An attempt to write a deeply nested match expressions to extract the underlying error is unfortunately brittle (especially when you want to make work generically) and a sign that the error you're looking at is an informative error, hence it is designed to be printed only for diagnostic purposes, but not to be matched on to influence program flow. Even if you obtained the underlying (informative) error struct as a result of nested match expressions, that error would likely be an boxed, opaque error, so the best you could do with it is to call something like .to_string() on it, which is almost no different from using DisplayErrorContext to print the whole chain of errors (admittedly, printing the whole chain is not as user-friendly as just printing the ServiceError { source: UnauthorizedException(UnauthorizedException { message: Some("Session token not found or invalid") portion, but I don't recall if there is a utility to extract just that part).

and only panic in case of an unkown reason that the code is not prepared to handle.

Even if DisplayErrorContext prints more than what you'd like to display to your user, it does include the information what the user needs to fix. What are the unknown reasons you're referring to? Is that specific to the configuration of your application (assuming you're writing a binary that uses aws-sdk-s3)?

[michaelrommel]

Thanks a lot for pointing me to the RFC - i did not know about their existence and now the design philosophy regarding error handling is much more clear to me. Yes, I agree that the needed information is present in the overly complicated nested error message and for now this needs to suffice.

I will then only try to handle the actionable errors from now on and let the others bubble up or present to the user as is.

Thank you very much for taking the time to clarifying this and sorry for the late reply.