ImageSharp.Web

[JimBobSquarePants]

Description

This issue is designed to capture the architecture and tasks required to build an extensible library for parsing URL API instructions and caching the result.

Feel free to discuss below. We'll assign specific issues for functional components as we determine them.

Basic Requirements

• Asp.NET Core Middleware to capture valid requests
• Extensible configuration for Caching, ImageServices, and Processing. Must be secure.
• IImageService implementations for physical file system and remote images.
• IDistributedCache implementation for physical file system that can be switched out for Redis, Azure, etc. How do we handle cleanup, Response codes?
• Fast, low collision hashing of URL for cache implementations.

Pipeline

The following gives an overall summary of the image processing pipeline.

1. Request is intercepted by Middleware (Authorisation?)
2. IImageService is assigned using URL prefix pattern matching (Regex?)
3. IImageService determines validity of request and attempts to return stream containing image.
4. IDistributedCache implementation checks if file exists. (Redirect if exists?)
5. Not cached? URL querystring params are parsed for matching processor.
6. Image is processed by matches via fluent API.
7. Result is cached and image served.
8. Cleanup of expired cached files. (How?, When?)

Edit

Updated Requirements and Pipeline below

Basic Requirements

• Asp.NET Core Middleware to capture valid requests
• Extensible configuration for Parsing, Resolving, Processing, and Caching. Must be secure
• Configuration for handling EXIF metadata
• IUriParser implementation for parsing URI requests for command string that can be switched out
• IImageResolver implementation collection for physical file system that
• Extensible CommandParser implementation for converting commands into real values for processing.
• IImageWebProcessor for Resize
• IImageWebProcessor for Quality
• IImageWebProcessor for Format
• IImageWebProcessor for BackgroundColor
• IImageCache implementation for physical file system that can be switched out for Azure, etc.
• Fast, low collision hashing of URL for cache implementations
• ILogger based logging
• TagHelper for generating img, picture markup

Pipeline

The following gives an overall summary of the image processing pipeline.

1. Request is intercepted by Middleware
2. IUriParser extracts commands; if not found, exit
3. IImageResolver is assigned using Match() method
4. IImageResolver determines validity of request and attempts to return stream containing image.
5. IImageCache implementation checks if file exists. (Returns correct response)
6. Not cached? URL querystring params are parsed for matching processor.
7. IImageWebProcessor implementations process the image
8. Result is cached and image served.

[clausjensen]

1. Authorization afaik shouldn't be ImageSharp's concern. You would plug in your authorization middleware before the ImageSharp.Web one and that would be responsible for handling anything authz-wise.

[JimBobSquarePants]

Authorization afaik shouldn't be ImageSharp's concern. You would plug in your authorization middleware before the ImageSharp.Web one and that would be responsible for handling anything authz-wise.

Great! I thought/hoped it would be the case.

[clausjensen]

Just adding this here for anyone wanting to get started: https://gist.github.com/clausjensen/bce60ccad6d65a92eadb4ea77229b287

[JimBobSquarePants]

@clausjensen I've actually started writing some code. Baby steps though.

https://github.com/JimBobSquarePants/ImageSharp/tree/e4d3aedb33af44bf8b384fa6a277b31bbfb2a381/src/ImageSharp.Web

[clausjensen]

Aaah awesome :) I didn't know how far you were on the basic middleware stuff but looks like you're already ahead of that!

[KLuuKer]

What about multi-tenancy and access restrictions to specific set's\folders of images?
for example: products that are not "live" yet only admins are allowed to access the images

[clausjensen]

@KLuuKer That would still not be the responsibility of ImageSharp.Web - that would be something you handle in your authorization middleware. ImageSharp should simply be responsible for handling requests that have already passed through any authorization middleware that may/may not allow the request to continue being handled.

[KLuuKer]

@clausjensen i can live with that

but allow to give prefixes or something for the cache keys, for multi-tenancy isolation (using the same cache)

[clausjensen]

@KLuuKer actually I think it may be a good idea to create a whole other issue just to talk about the multi tenancy stuff, as I'm pretty certain there's a lot of things in regards to how caching should/could be done and how caching can best be configured for these kind of setups.

I'm a bit worried notes and ideas like these may get lost if not separated out into an issue focused just on this part. Unfortunately it is not something I know that much about so I suspect there's a lot of potential pitfalls here we might as well get addressed to begin with. ( @JimBobSquarePants ?)

[JimBobSquarePants]

@clausjensen @KLuuKer I'll set up issues for each item later tonight.

Re caching, that's what IDistributedCache is for. Want multitenancy, switch to Azure caching or another implementation.

[lfoust]

I think a good place to start in designing the middleware is to determine what the inputs to the middleware would be and how those come into the request (query string, headers, etc). Also what are the possible operations (is it just for resizing or are there other processing operations supported?)

It would be great if the mapping between the inputs and how they make it to the middleware was configurable so that people can use whatever routing or conventions they want and the image middleware would handle the heavy lifting.

For example, some inputs for resizing could be:

• Width - this would likely come from the url
• Height - this would likely come from the url
• Quality - this would likely come from the url
• Format - this could come from an accept header on the request
• Focal Point - this would likely come from the url

Are there any existing image services or plugins that have functionality you would like to incorporate? Are there any standards around such services?

Coming up with some sample scenarios like:

/images?w=100&h=250&q=.8
/images/100/250/.8

and walking through how the request would flow would be a great exercise. Also including caching and authentication/authorization in these scenarios will help spell out how those pieces interact.

Another big consideration in the design is making sure the response is property formatted to be consumed by a CDN. It is great to support IDistrubutedCache but ultimately CDNs are the best and most common way to cache images.

[romain-preston]

2. IImageService is assigned using URL prefix pattern matching (Regex?)

Should IImageService interface something like Task<bool> ShouldProcessAsync(HttpContext context)
so the IImageService is responsible for its own implementation of URL matching.

On step 5 (Not cached? URL querystring params are parsed for matching processor.) either there is a global parser that fills some kind of ResizeParameter Dictionary or again each IImageService get what he needs from the Request. It would be a solution to implement different resizing parameter conventions (headers, querystring, path…)
I think a global IImageServiceParameterProvider could be helpful to separate well the IImageService from its parameters needs.
If so the Task<bool> ShouldProcessAsync(HttpContext context) would become something like Task<bool> ShouldProcessAsync(IImageServiceParameterProvider provider) ???

Another consideration:
it would be helpfull to design the IImageServices in a chaining fashion…
We could then imagine something like :
AzureStoreImageService => ImageManipulationService

This case Task<Stream> ResolveImageAsync(HttpContext context, string path) is maybe not the best interface for it…

It could maybe also help to make preventive imageservice like :
PreventDdoSAttackImageService => ImageManipulationService

[JimBobSquarePants]

@lfoust

It would be great if the mapping between the inputs and how they make it to the middleware was configurable so that people can use whatever routing or conventions they want and the image middleware would handle the heavy lifting.

That feels to me like opening a can of worms. I would stick with a single unified URL Querystring API with identifiers for IImageService implementations using a prefix.

e.g. http://mysite.com/remote/http://remote-image.jpg?width=200 would resolve to the RemoteImageService since we are using the remote prefix.

If an IImageService had an empty prefix e.g. PhysicalFileImageService it would match all image requests containing recognized querystring parameters. Any implementations like this would be a last resort and only called when there are no prefix matching services.

All the inputs you mentioned would come from the Querystring API

• width=200
• height=100
• quality=75
• format=png
• xy=20&xy=50 (This is the standard method to pass arrays through qs params, though very verbose)

Note IMO It's better to use more descriptive parameters as we would support more processors as plugins.

There are no standards around this as far as I'm aware.

Another big consideration in the design is making sure the response is property formatted to be consumed by a CDN. It is great to support IDistrubutedCache but ultimately CDNs are the best and most common way to cache images.

Exactly. My thought would be that use the file system IDistrubutedCache for smaller sites but if you want scaling you would use an Azure or Amazon based implementations (This is something I would probably build privately and sell licenses for - Gotta earn something from my years or work!). Those caches would be installed on a custom endpoint that the CDN will request if no matching image is found on their system.

@romain-preston

Each IImageService should definitely have something like Task<bool> ShouldProcessAsync(HttpContext context) though I think we would have already split the request URL into different parts for processing so would pass them to the method.

On Step 5. Each installed processor would accept and parse the querystring NameValueCollection and return a result accordingly. All instruction sets are querystring only. Every CDN I know allows caching by querystring params.

Another consideration:
it would be helpfull to design the IImageServices in a chaining fashion…
We could then imagine something like :
AzureStoreImageService => ImageManipulationService

I don't see any benefit in this. Each service is responsible for matching a pattern and returning an image. Chaining would only add overheads and complexity.