use urlcache

[stevenvachon]

You can offload the logic elsewhere, which also supports normalization to avoid duplicates.

[stevenvachon]

But honestly, I think that offering cache in general to this library is bloat.

[ForbesLindesay]

Caching is the main point of this library. This library is a caching http client that supports redirects, that's it.

Urlcache doesn't look like it solves a problem we actually have:

1. I'm not sure we should need to normalise urls in the cache.
2. We need a file system implementation, not just an in-memory implementation.
3. We need an efficient way to only load the body into memory if all the header information was a match.

What we have works fine. Correct caching is the main problem that's difficult and solved by this library.

[stevenvachon]

"caching" is not mentioned in the library description. All it says is "a simple wrapper". Caching is going beyond making a request.

[derhuerst]

I'm not sure we should need to normalise urls in the cache.

I'm trying to cache assets attached to a GitHub release, which are being served via AWS. I get URLs with varying query parameters:

https://github-production-release-asset-2e65be.s3.amazonaws.com/29458513/26341680-4231-11ea-8e36-ae454621d74a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200405%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200405T161319Z&X-Amz-Expires=300&X-Amz-Signature=25da7ad2122fe44a87a6e42565d066277c603080814f5c99f273aa29e4d7c527&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Ddarwin-x64&response-content-type=application%2Foctet-stream
https://github-production-release-asset-2e65be.s3.amazonaws.com/29458513/26341680-4231-11ea-8e36-ae454621d74a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200405%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200405T163811Z&X-Amz-Expires=300&X-Amz-Signature=c4e8b2bfdf7631caab495cbd276337589becd3a4f9cd7b4f6684961b0fddfc7a&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Ddarwin-x64&response-content-type=application%2Foctet-stream

For the caching to work, I need to exclude the AWS S3 signature parameters:

• X-Amz-Algorithm
• X-Amz-Credential
• X-Amz-Date
• X-Amz-Expires
• X-Amz-Signature
• X-Amz-SignedHeaders

Unfortunately, http-basic calls cache.getResponse directly without allowing me to transform the URL, and FileCache uses the internal function getCacheKey directly.

Would you mind a PR that moves getCacheKey to the prototype of FileCache, so that people can overwrite it?