Description
openedon Sep 13, 2023
Description
To get an access token, AzureCliCredential
shells out to az account get-access-token
and parses up the expiresOn
string as a datetime value. Since az
formats the datetime without a timezone offset, it assumes the offset with date::assume_local()
.
date::assume_local()
delegates to time
's function UtcOffset::current_local_offset()
.
azure-sdk-for-rust/sdk/core/src/date/mod.rs
Lines 116 to 119 in f978185
For most Unix systems, UtcOffset::current_local_offset()
will return Ok(offset)
for single-threaded applications, and Err(IndeterminateOffset)
for multi-threaded applications, as defined by this function.
In my case, I am running my application on Ubuntu (WSL), and my application is multi-threaded. So this offset is always indeterminate.
(Aside: There's a whole conversation about thread-safety of accessing local time zone data, which is the reason why this behavior exists. If certain things change in the Rust ecosystem, then this problem may just resolve itself as simply as installing a new release of time
.)
Because the offset is indeterminate, date::assume_local()
will default the expiry time to being in the UTC time zone, when it should be in my local (PDT) time zone. This is obviously incorrect behavior, because the expiry time is now hours off from the actual time it should represent.
This can manifest as bugs further down the line, e.g. in AutoRefreshingTokenCredential
where recently cached tokens are refreshed constantly because they are viewed as expired. Each token refresh adds consistent overhead to SDK calls. In particular, this led my calls through SDK clients to take approximately ~1.3s each rather than an expected ~90ms each when the token was only refreshed as needed.
Workarounds/Solutions
- As users of the SDK, what we can do to mitigate this:
- (RECOMMENDED) Use other credential types. This will avoid the issue, simply because other credential types don't interact with this code.
- (NOT recommended) Manually override the thread-count check by using the
set_soundness()
function provided bytime
. This requires some understanding of the thread-safety of your application and probably isn't worth the effort to do correctly.
- As SDK authors, what we can do to fix this:
- Wait for the Rust ecosystem to figure out a way to manage thread-safe access of local time information, then consume this new thread-safe interface.
- Push for changes in the Azure CLI output. It seems az-cli was looking into representing the expiry time as an integer timestamp instead (
az account get-access-token
: Use epochexpiresOn
/expires_on
azure-cli#19700). If implemented, theAzureCliCredential
would just have to deserialize the epoch timestamp, circumventing the need to determine the system's local offset. However, I haven't seen much activity on this issue, so I'm not sure what the timelines are there. - We could also try to use
set_soundness()
in the SDK itself, but this is probably unsafe. Again, not recommended.