Add an S3 key/value storage provider interface

[JonathanWoollett-Light]

There exists the capability to use a different storage backend for the different key/value stores e.g. https://developer.fermyon.com/spin/v2/dynamic-configuration#libsql-storage-provider.

A key aspect of many applications is object storage for cheap storage of large rarely edited binary data (e.g. photos).

The costs that make the current system untenable are shown below.

Fermyon key/value storage

Based off the Growth plan from https://www.fermyon.com/pricing at $19.39/month for 2GB of key value storage. This works out to:

$19.39 / 2^30 bytes
= $0.00000001805834472179412841796875 per byte 
= $0.0000184917449951171875 per kb
= $0.018935546875 per mb

Assuming each user uploads 4 high quality images at 4 mb per image this is $0.30296875 per user per month. With 1,000 users this is $302.96875 per month.

AWS S3

Based off https://aws.amazon.com/s3/pricing/ at $0.023 per GB per month. This works out to $0.0000224609375 per mb. Which at 4 high quality images at 4mb per image this is $0.000359375 per user. With 1,000 users this is $0.359375 per month.

[macolso]

Thanks for providing feedback, @JonathanWoollett-Light! Appreciate you taking the time to share your calculations. One key difference in my mind is that the Fermyon Cloud Growth price includes many resources such as KV storage, SQLite storage, egress, and compute whereas the pricing for AWS S3 is limited to just object storage. Do you mind sharing a bit more about your use case (i.e., beyond cost, are there additional reasons why you would prefer an AWS S3-backed KV store for your Spin applications) so we can best triage this ask?

[JonathanWoollett-Light]

One key difference in my mind is that the Fermyon Cloud Growth price includes many resources such as KV storage, SQLite storage, egress, and compute whereas the pricing for AWS S3 is limited to just object storage.

Yes, it is not a fair comparison, but without specific pricing for key/value storage it was not possible to give a better comparison.

Do you mind sharing a bit more about your use case (i.e., beyond cost, are there additional reasons why you would prefer an AWS S3-backed KV store for your Spin applications) so we can best triage this ask?

Storing photos a user may upload.

[macolso]

Great, thanks for that context. I think there are two parts here:

1. Support AWS S3 as a Spin Key Value store provider
2. Support runtime configuration on Fermyon Cloud (today it's a limitation)

If you don't mind, I will transfer this issue to the Spin repo for Point 1 so it can be triaged appropriately. As for a workaround in the meantime, there are members of the Spin community who have connected their Spin apps to S3 using allowed outbound HTTP who'd likely be happy to chat about their experience (Discord thread). Folks from Fermyon would also be happy to brainstorm on the topic as well!