Don't run page middleware on requests served from /public folder

[goleary]

Describe the feature you'd like to request

We use the same next.js app for both our marketing site and for our authenticated webapp.

We host our marketing site at blah.com & our app at app.blah.com.

We have all of our "app" pages in pages/app. We also have root _middleware.ts that rewrites all requests from app.blah.com=>blah.com/app.

The setup is lovely, the only wrinkle is that we share public assets between these different sections of our app. We have stuff like:
public/font.css. From the app.blah.com subdomain the request for these resources fail because the middleware runs and looks for the resources at public/app/font.css.

Describe the solution you'd like

I'd love it if any resource requests that are filled from the /public directory would not pass through the page middleware. They aren't page requests, so why do they go through the page middleware?

Describe alternatives you've considered

If I duplicate all of my resources from /public=>/public/app it avoids the 404s because once the middleware has rewritten the path it can still find the resources. This is suboptimal because it requires duplicating files.

I can put all resources in a sub folder like /public/resources & then in the root page middleware I can avoid rewriting requests prefixed by /resources. This is suboptimal because we have files that are expected to be in specific locations on our root path (i.e. favicon.ico, /.well-known/appple-app-site-association).

I can handle each file/folder present in /public in my /pages/_middleware.ts & avoid rewriting - this is suboptimal because it requires a lot of extra code & is error prone because if anything in that folder changes or if a root level file is added it won't be accessible.

Any other options?

[Dragate]

I don't mind the middleware running on public assets, however I'd love if the middleware provided a boolean flag whether the request is for a public asset. Currently I'm using the solution from this example, which basically checks if the url has a file extension. This mostly works, but if your app also serves protected files outside the public folder (e.g. generating .pdf file on demand), you'll have to add extra logic in middleware to handle those files differently.

[icyJoseph]

I know it might not be ideal, but you could do this?

function shouldExclude(request: NextRequest) {
    const path = request.nextUrl.pathname;

    return (
        path.startsWith('/api') || //  exclude all API routes
        path.startsWith('/static') || // exclude static files
        path.includes('.') // exclude all files in the public folder
    );
}

I think for A/B testing, logging, and a bunch of other things, it is good that public is taken into account.

Perhaps least know, but similarly, placing a [...slug].tsx at the top of pages, will also attempt to catch requests coming to public, if the file doesn't exist. For example, if there was not favicon.ico in public, it'd arrived at [...slug].tsx and slug would be ['favicon.icon'].

[goleary]

path.includes('.') is clever and works! Thanks for the suggestion, I hadn't thought of that.

Agreed, it's not ideal, but will do for now.

Regarding the fallbackSlug at the top of pages, yeah we use that, but thankfully it only catches requests that can't be served from public which is exactly what I want _middleware.ts to do.

[VladimirMikulic]

In case anyone else is browsing this thread now that middleware is stable (Next.js v12.2.0+), here's matcher solution for public assets/api routes/_next exclusion:

export const config = {
  matcher: "/((?!api|static|.*\\..*|_next).*)",
};

[orbulant]

For anyone using Next.js 13 with next-intl this is what worked for me when creating Middlewares.

[Stephen-Strange]

Same here.. NextJS 13 + i18n.. Had to give .*\\..* for certain images to load.
But why?
What does .*\\..* do?

[VladimirMikulic]

@Stephen-Strange .*\\..* matches "url.extension" so in your case it excludes images.

[DEBorodina]

worked for me too (NextJS 13 + i18n)
I followed these:
https://locize.com/blog/next-13-app-dir-i18n/
in middleware.ts replace
matcher: ['/((?!api|_next/static|_next/image|assets|favicon.ico|sw.js).)']
with
matcher: [
'/((?!api|.\..|_next/static|_next/image|assets|favicon.ico|sw.js).)',
],
might be useful to someone

[zelezekas]

The downside of this approach is that when the URL does contain a dot but we don't host that specific file in the public directory, the server returns a 500 error instead of a 404 page. Is there a solution for that?

[gabrielmaldi]

robots.txt should probably be ignored too:

export const config = {
  matcher: [
    /*
     * Match all request paths except for the ones starting with:
     * - api (API routes)
     * - _next/static (static files)
     * - _next/image (image optimization files)
     * - favicon.ico (favicon file)
     * - robots.txt (robots file)
     */
    "/((?!api|_next/static|_next/image|favicon.ico|robots.txt).*)",
  ],
};

https://nextjs.org/docs/app/building-your-application/routing/middleware#matcher

[yanickrochon]

When I add this rule, then the middleware is never called at all! And if I add "/:path*" then all requests go through the middleware.

Why isn't there an "exclude" option instead of having these negative matching dumb rules?

[MoltenCoffee]

A 'proper' solution from Vercel would be appreciated, even the examples in the docs lead to this problem appearing on the starter project.

At the moment, my solution is to have a /public subdirectory within /public, so my assets are at /public/public. Horrific to look at in the file explorer, but when referencing files the added /public looks quite natural, and does not come with the problems described by others when ignoring all paths with a . in them.

My current regex is as follows:

export const config = {
  matcher: [
    "/((?!_next/static|_next/image|public/|favicon.ico|robots.txt|sitemap.xml|manifest.json).*)",
  ],
};

[nicolasjrz]

me funciono 10/10,

[justinh00k]

Using next-intl middleware, if I don't specifically exclude all public routes, they 404. Not easy to notice something like robots.txt not loading and can obviously have a deleterious effect on SEO.

But having to list all root level public files (favicon.ico|robots.txt|sitemap.xml|manifest.json|ads.txt....) is not ideal, and there are cases (like say, auto generated sitemaps) where you may not know exactly the names of the files.

Here's what I'm working with currently:
'/((?!api|_next/static|_next/image|favicon.ico|robots.txt|ads.txt|sitemap.xml|manifest.json|android-chrome-192x192.png|apple-touch-icon.png|browserconfig.xml|mstile-150x150.png|safari-pinned-tab.svg|site.webmanifest|favicon-.*.png).*)',

[jd-carroll]

I agree that this is super frustrating and that there should be a flag that indicates the request will be handled by public. However...

That is actually entirely impossible given the current request handling architecture in Next. If we use the Middleware documentation as an example, middleware is executed early to allow redirect / rewrites (step 3). Next does not check the file system until step 5 where it attempts to handle static assets.

So if anything, supporting a "public" flag would need to be in the opposite direction, something like: "prefix this route, unless it can be handled by /public" ... But that doesn't really make sense and would likely introduce a lot of complexity.

Alternate Solution

Instead what I did was embrace the locale. Here is the structure of the solution:

• No files of any kind in the root of /public
• Create a single folder for your default locale (e.g. en_US) and place all of the standard static files there (favicon.ico, robots.txt, ...)
• Write a pre-build script which creates a copy of the default locale assets for every locale that you support (you can write it such that it only does this in CI equivalent environments)

Now, you can use a simple matcher like: '/((?!api|_next|_vercel).*)'
And not have to worry about introducing unexpected bugs because one of your paths happens to contain a ..

Further, if you did have static assets that you store under /public you would continue to operate those in the same way. Meaning, they would exist outside of the locale folder under /public and you would update your matcher to be something like: '/((?!api|_next|_vercel|static).*)'.

Last thought, while expanding the matcher to include more files does approach a "cleaner" solution, there is a performance tradeoff as the complexity of the RegExp grows. While maybe less "clean", I think this approach has the correct balance of organization and performance. Plus assets like site.webmanifest should arguably be contained in a locale specific folder.

Hope this helps.

[beforedisappear]

Using this regex "/((?!api|_next/static|_next/image|favicon.ico).*)" from the documentation, I find a bad UI at addresses such as "/favicon.icoo" , "robots.txtt" or others. App using minimal i18n rouring w/o external libs. Default locale is hidden by rewrite from NextResponse.

[theenoahmason]

Just chiming in as I'm running into this as well in 2024. For the time being, adding subdirectories to /public seems to be the cleanest way to handle large batches of files. ie: public/images/....

export const config = {
  matcher: [
    /*
     * Match all request paths except for the ones starting with:
     * - api (API routes)
     * - _next/static (static files)
     * - _next/image (image optimization files)
     * - images directory in /public (public static images)
     */
    '/((?!api|_next/static|_next/image|images).*)',
  ],
};

• root level files can still be targeted (or not) without having to target every individual asset.
• More elegant than using .*\\..* which is way too catch-all (and has its own issues as described above).
• Bonus: encourages organization? 🤭

[zahitrios]

Tested in nextjs 15 and it works!

[hafizusama43]

I am using clerk and this is recommended in their documentation. Hope it helps

export const config = {
  matcher: [
    // Skip Next.js internals and all static files, unless found in search params
    '/((?!_next|[^?]*\\.(?:html?|css|js(?!on)|jpe?g|webp|png|gif|svg|ttf|woff2?|ico|csv|docx?|xlsx?|zip|webmanifest)).*)',
    // Always run for API routes
    '/(api|trpc)(.*)',
  ],
};

[josephgoksu]

This one still doesn't work.

[optonpo]

to clarify: is middleware regex matching next's best/only option for creating public routes? my mind is mush after spending 6 hours trying nearly every forum suggestion.

edit: if so, shouldn't something like '/((?!_next/static|_next/image|favicon.ico|home(/.)?|.\.(?:svg|png|jpg|jpeg|gif|webp)$).*)' bypass auth for https://example.com/home?

[qtwf]

My site doesn't even need middleware on any served files, so I can go rogue and just exclude all URLs that have dots in them:

export const config = {
  matcher: [
    '/((?!api/|test/|.*[.]).*)',
  ],
}

or /((?!api/|test/|.*[^/][.]).*) if you dont want to target paths that have dots in the beginning (e.g. /page/.secret/page)