Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement AWS key value store #2883

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ogghead
Copy link
Contributor

@ogghead ogghead commented Oct 10, 2024

Hi folks! I am creating this draft PR to solicit feedback on an initial AWS key value store implementation. I appreciate any and all discussions on this PR!

Some points for thought:

  1. Implementation uses DynamoDB, though for large blob storage, S3 is preferable (DynamoDB can only store <=400KB size records), see Add an S3 key/value storage provider interface #2606. DynamoDB is cheaper and faster for performing many rapid reads/writes of small amounts of data though, and is roughly in the same niche as Azure CosmosDB
  2. Auth currently requires generating AWS STS token credentials and passing them to the Spin app in a runtime config file. https://github.com/spinkube/skips/pull/9/files discusses better patterns to fetch credentials. Curious to hear thoughts on how this implementation can integrate better with that proposal!
  3. The Azure key-value implementation supports reading credentials from environment variables, however the AWS Rust SDK does not offer a synchronous API to load config and would require the MakeKeyValueStore::make_store function to be async for all implementations -- leading to a chain of async function coloring. It is possible to manually fill the SdkConfig object and I did this to pass STS tokens from a runtime config file, but it would be ideal to rely on the SDK's defaults and many credential loading fallbacks if possible. Curious on thoughts for how to best handle env var credential loading

Signed-off-by: Darwin Boersma <darwin@sadlark.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant