Support wildcard patterns for upstream remote images

[styfle]

Abstract

Add support for wildcard patterns when configuring the allowlist for upstream remote images.

Background

We currently support a list of domains in next.config.js to configure which hosts are allowed to serve source images that can then be optimized by Next.js. Without this restriction, someone could hot-link to your Image Optimization API with their images instead of yours website’s images and you would have to eat the cost of optimizing their images.

Problem We Need To Solve

The domains implementation is too restrictive because it only allows an exact match on the hostname. It doesn’t allow subdomains or any path matching. So even allowing a single external image linking to social network or CMS would open up the door for any image hosted on that domain to be allowed to optimize.

We would like to solve the following use cases:

• **.fbcdn.net - Match all subdomains recursively (Facebook Avatars)
• avatar-management--avatars.*.prod.public.atl-paas.net - Match only part of the domain which represents a region (Bitbucket Avatars)
• www.datocms-assets.com/13375P34K/** - Match domain but also match all paths after the account number (DatoCMS)
• https://assets.vercel.com - Match protocol and domain (today we allow both http/https)

Proposed Solution

Provide a new property, remotePatterns, and deprecate domains. This would still support exact matches to cover the use case of domains, but it would also provide a way to define wildcard subdomains, path patterns, or even be extended in the future to support other parts of the URL.

module.exports = {
  images: {
    remotePatterns: [
      // exact match `example.com`
      { hostname: 'example.com' },
      // wildcard subdomains to match all facebook avatars
      { hostname: '**.fbcdn.net' },
      // wildcard the region to match all bitbucket avatars
      { hostname: 'avatar-management--avatars.*.prod.public.atl-paas.net' },
      // exact match hostname but wildcard path to match the account
      { hostname: 'www.datocms-assets.com', pathname: '/13375P34K/**' },
      // exact match protocol and domain
      { protocol: 'https', hostname: 'assets.vercel.com' },
    ]
  }
}

The parts of the URL that are not defined would implicitly be considered wildcards. For example, since query is not defined, it could match anything.

We should require at least hostname is defined, while other props can be optional.

We’ll also want to impose a maximum limit to the array, so we’ll start with 50 to match the current limit of domains.

Alternatives Considered

• Reusing the domains property could be nice because users know to look there already, however the name doesn’t make sense once pathname or protocol is included and its not clear it supports pattern matching.
• Using the full URL string, such as https://www.datocms-assets.com/13375P34K/**, instead of breaking it into parts would make for a nicer experience but it would lead to ambiguity. For example, what if the user wanted to match port or querystring in the future? Wildcards are placeholders so its not clear which part of the URL should be a pattern and which part should be literal.
• Reusing the existing pattern matching syntax from [rewrites](https://nextjs.org/docs/api-reference/next.config.js/rewrites), such as https://www.datocms-assets.com/13375P34K/:path*. This would work great if we can utilize the same syntax for host but that could be challenging since the host uses a . separator as opposed to /.
• Using PCRE (regex) would be verbose and prone to error because . (and many other chars) need to be escaped.
• Using a rewrite is possible today for some of these use cases but not all (doesn't work for hostname patterns).
• Using the new URLPattern object but its still experimental and subject to change.

Related

• Allow a wildcard for images.domains in next.config.js / ability to disable Next/Image hostname check #18429
• Allow image optimization on wildcard hostnames #18632
• Add support for wildcard patterns in image.domains #18730
• Allow wildcarded domains for image optimization #27345

[FDiskas]

Hi,
Thanks for proposal.
What about this one https://github.com/isaacs/minimatch

NPM internally use that. So well tested and documented and also will be supported in the future

[styfle]

Does it work with periods instead of forward slashes?

I think the best solution is to write our own parser since we’ll have full control and can extend it later. I would imagine we need to add support for hostname patterns and pathname patterns, but its not clear if the port matters for example.

Also, we don't want to inherit any vulnerabilities of minimatch if we don't have to.

[neeraj3029]

Hi @styfle @FDiskas, any plans of going ahead with this?