Refactor prerendering chunk handling

[bluwy]

Changes

NOTE: This is a big PR but mostly due to test changes/additions.

This PR changes how chunks are handled for prerendering. Before:

• For each page, the page.route.prerender metadata is derived in manualChunks (slow)
• If page.route.prerender is true, it's grouped into a special "prerender" chunk.
• After prerendering, we search for the "prerender" chunk and mark every export with a no-op.
• PROBLEM: The "prerender" chunk may contain non-prerender code due to Rollup chunking, where it thinks certain code are used together and group them in a single chunk regardless.

After:

• For each page, the page.route.prerender metadata is derived in generateBundle (faster)
• During generateBundle, we figure out chunks that are used in prerender-only, and track them to be deleted at the end.
• To make the chunk deletion efficient, each page has their corresponding Rollup input where we can be sure of deleting the input directly after building.

Testing

It includes test from #10738

There are also a lot of changes related to outDir, because each page are now its own input chunk, when we build fixtures multiple times, ESM could cache the input chunk path, causing subsequent fixtures runtime to use the previous, which is problematic also for CCC.

Docs

n/a. Mostly a bug fix and it shouldn't affect Astro projects in practice.

[changeset-bot]

🦋 Changeset detected

Latest commit: d0bd620

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[bluwy]

Left some review notes below

[ematipico]

Awesome work @bluwy !

[bluwy]

!preview refactor-prerender

[github-actions]

Snapshots have been released for the following packages:

• astro@experimental--refactor-prerender
• @astrojs/vercel@experimental--refactor-prerender

Publish Log

🦋  warn ===============================IMPORTANT!===============================
🦋  warn Packages will be released under the experimental--refactor-prerender tag
🦋  warn ----------------------------------------------------------------------
🦋  info npm info astro
🦋  info npm info @astrojs/prism
🦋  info npm info @astrojs/rss
🦋  info npm info create-astro
🦋  info npm info @astrojs/db
🦋  info npm info @astrojs/alpinejs
🦋  info npm info @astrojs/lit
🦋  info npm info @astrojs/markdoc
🦋  info npm info @astrojs/mdx
🦋  info npm info @astrojs/node
🦋  info npm info @astrojs/partytown
🦋  info npm info @astrojs/preact
🦋  info npm info @astrojs/react
🦋  info npm info @astrojs/sitemap
🦋  info npm info @astrojs/solid-js
🦋  info npm info @astrojs/svelte
🦋  info npm info @astrojs/tailwind
🦋  info npm info @astrojs/vercel
🦋  info npm info @astrojs/vue
🦋  info npm info @astrojs/web-vitals
🦋  info npm info @astrojs/internal-helpers
🦋  info npm info @astrojs/markdown-remark
🦋  info npm info @astrojs/studio
🦋  info npm info @astrojs/telemetry
🦋  info npm info @astrojs/underscore-redirects
🦋  info npm info @astrojs/upgrade
🦋  info astro is being published because our local version (0.0.0-refactor-prerender-20240614140807) has not been published on npm
🦋  warn @astrojs/prism is not being published because version 3.1.0 is already published on npm
🦋  warn @astrojs/rss is not being published because version 4.0.6 is already published on npm
🦋  warn create-astro is not being published because version 4.8.0 is already published on npm
🦋  warn @astrojs/db is not being published because version 0.11.5 is already published on npm
🦋  warn @astrojs/alpinejs is not being published because version 0.4.0 is already published on npm
🦋  warn @astrojs/lit is not being published because version 4.2.0 is already published on npm
🦋  warn @astrojs/markdoc is not being published because version 0.11.0 is already published on npm
🦋  warn @astrojs/mdx is not being published because version 3.1.0 is already published on npm
🦋  warn @astrojs/node is not being published because version 8.3.0 is already published on npm
🦋  warn @astrojs/partytown is not being published because version 2.1.1 is already published on npm
🦋  warn @astrojs/preact is not being published because version 3.4.0 is already published on npm
🦋  warn @astrojs/react is not being published because version 3.5.0 is already published on npm
🦋  warn @astrojs/sitemap is not being published because version 3.1.5 is already published on npm
🦋  warn @astrojs/solid-js is not being published because version 4.3.0 is already published on npm
🦋  warn @astrojs/svelte is not being published because version 5.5.0 is already published on npm
🦋  warn @astrojs/tailwind is not being published because version 5.1.0 is already published on npm
🦋  info @astrojs/vercel is being published because our local version (0.0.0-refactor-prerender-20240614140807) has not been published on npm
🦋  warn @astrojs/vue is not being published because version 4.4.0 is already published on npm
🦋  warn @astrojs/web-vitals is not being published because version 0.2.1 is already published on npm
🦋  warn @astrojs/internal-helpers is not being published because version 0.4.0 is already published on npm
🦋  warn @astrojs/markdown-remark is not being published because version 5.1.0 is already published on npm
🦋  warn @astrojs/studio is not being published because version 0.1.0 is already published on npm
🦋  warn @astrojs/telemetry is not being published because version 3.1.0 is already published on npm
🦋  warn @astrojs/underscore-redirects is not being published because version 0.3.3 is already published on npm
🦋  warn @astrojs/upgrade is not being published because version 0.3.1 is already published on npm
🦋  info Publishing "astro" at "0.0.0-refactor-prerender-20240614140807"
🦋  info Publishing "@astrojs/vercel" at "0.0.0-refactor-prerender-20240614140807"
🦋  success packages published successfully:
🦋  astro@0.0.0-refactor-prerender-20240614140807
🦋  @astrojs/vercel@0.0.0-refactor-prerender-20240614140807
🦋  Creating git tags...
🦋  New tag:  astro@0.0.0-refactor-prerender-20240614140807
🦋  New tag:  @astrojs/vercel@0.0.0-refactor-prerender-20240614140807

Build Log

> root@0.0.0 build /home/runner/work/astro/astro
> turbo run build --filter=astro --filter=create-astro --filter="@astrojs/*" --filter="@benchmark/*"

• Packages in scope: @astrojs/alpinejs, @astrojs/cloudflare, @astrojs/db, @astrojs/internal-helpers, @astrojs/lit, @astrojs/markdoc, @astrojs/markdown-remark, @astrojs/mdx, @astrojs/netlify, @astrojs/node, @astrojs/partytown, @astrojs/preact, @astrojs/prism, @astrojs/react, @astrojs/rss, @astrojs/sitemap, @astrojs/solid-js, @astrojs/studio, @astrojs/svelte, @astrojs/tailwind, @astrojs/telemetry, @astrojs/underscore-redirects, @astrojs/upgrade, @astrojs/vercel, @astrojs/vue, @astrojs/web-vitals, @benchmark/timer, astro, create-astro
• Running build in 29 packages
• Remote caching enabled
::group::@astrojs/telemetry:build
cache hit, suppressing logs 087d8ef0d2c1a399
::endgroup::
::group::@astrojs/internal-helpers:build
cache hit, suppressing logs a304b24d43af2790
::endgroup::
::group::@astrojs/markdown-remark:build
cache hit, suppressing logs c990d2b3884cf01b
::endgroup::
::group::@astrojs/prism:build
cache hit, suppressing logs 61f70c3cc32fdd90
::endgroup::
::group::create-astro:build
cache hit, suppressing logs a79ef3399ce64fd0
::endgroup::
::group::@astrojs/upgrade:build
cache hit, suppressing logs e97cdb908e0f028d
::endgroup::
::group::astro:build
cache miss, executing 35d9ad82218194e6

> astro@0.0.0-refactor-prerender-20240614140807 build /home/runner/work/astro/astro/packages/astro
> pnpm run prebuild && astro-scripts build "src/**/*.{ts,js}" && tsc && pnpm run postbuild


> astro@0.0.0-refactor-prerender-20240614140807 prebuild /home/runner/work/astro/astro/packages/astro
> astro-scripts prebuild --to-string "src/runtime/server/astro-island.ts" "src/runtime/client/{idle,load,media,only,visible}.ts"


> astro@0.0.0-refactor-prerender-20240614140807 postbuild /home/runner/work/astro/astro/packages/astro
> astro-scripts copy "src/**/*.astro" && astro-scripts copy "src/**/*.wasm"

::endgroup::
::group::@astrojs/lit:build
cache miss, executing 6e2b196fab4d4758

> @astrojs/lit@4.2.0 build /home/runner/work/astro/astro/packages/integrations/lit
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/alpinejs:build
cache miss, executing 9b438d196c8ff412

> @astrojs/alpinejs@0.4.0 build /home/runner/work/astro/astro/packages/integrations/alpinejs
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@benchmark/timer:build
cache miss, executing 74d47b1307df11b3

> @benchmark/timer@0.0.0 build /home/runner/work/astro/astro/benchmark/packages/timer
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/react:build
cache miss, executing 67fed34848774501

> @astrojs/react@3.5.0 build /home/runner/work/astro/astro/packages/integrations/react
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/mdx:build
cache miss, executing b59271100633ee75

> @astrojs/mdx@3.1.0 build /home/runner/work/astro/astro/packages/integrations/mdx
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/node:build
cache miss, executing b1736a541c87f468

> @astrojs/node@8.3.0 build /home/runner/work/astro/astro/packages/integrations/node
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/markdoc:build
cache miss, executing 98ee6ea1fcde5d6d

> @astrojs/markdoc@0.11.0 build /home/runner/work/astro/astro/packages/integrations/markdoc
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/preact:build
cache miss, executing 8b40af4e740c28f5

> @astrojs/preact@3.4.0 build /home/runner/work/astro/astro/packages/integrations/preact
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/svelte:build
cache miss, executing c7485f491f9a9757

> @astrojs/svelte@5.5.0 build /home/runner/work/astro/astro/packages/integrations/svelte
> astro-scripts build "src/index.ts" && astro-scripts build "src/editor.cts" --force-cjs --no-clean-dist && tsc

::endgroup::
::group::@astrojs/vue:build
cache miss, executing e5fbb0e14f3da417

> @astrojs/vue@4.4.0 build /home/runner/work/astro/astro/packages/integrations/vue
> astro-scripts build "src/index.ts" && astro-scripts build "src/editor.cts" --force-cjs --no-clean-dist && tsc

::endgroup::
::group::@astrojs/studio:build
cache miss, executing df446708b12951ce

> @astrojs/studio@0.1.0 build /home/runner/work/astro/astro/packages/studio
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/partytown:build
cache miss, executing 262f354226e34f0e

> @astrojs/partytown@2.1.1 build /home/runner/work/astro/astro/packages/integrations/partytown
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/tailwind:build
cache miss, executing 2e124d27be008050

> @astrojs/tailwind@5.1.0 build /home/runner/work/astro/astro/packages/integrations/tailwind
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/rss:build
cache miss, executing 419f68c6f27f15d9

> @astrojs/rss@4.0.6 build /home/runner/work/astro/astro/packages/astro-rss
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/solid-js:build
cache miss, executing d4bd0ed0c37a5524

> @astrojs/solid-js@4.3.0 build /home/runner/work/astro/astro/packages/integrations/solid
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/underscore-redirects:build
cache miss, executing a83e66798593ea3d

> @astrojs/underscore-redirects@0.3.3 build /home/runner/work/astro/astro/packages/underscore-redirects
> astro-scripts build "src/**/*.ts" && tsc -p tsconfig.json


> @astrojs/underscore-redirects@0.3.3 postbuild /home/runner/work/astro/astro/packages/underscore-redirects
> astro-scripts copy "src/**/*.js"

::endgroup::
::group::@astrojs/vercel:build
cache miss, executing 6be481cfc951a104

> @astrojs/vercel@0.0.0-refactor-prerender-20240614140807 build /home/runner/work/astro/astro/packages/integrations/vercel
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/sitemap:build
cache miss, executing cb47ecee496b56cf

> @astrojs/sitemap@3.1.5 build /home/runner/work/astro/astro/packages/integrations/sitemap
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/db:build
cache miss, executing 95dedfc1e3ba4dfe

> @astrojs/db@0.11.5 build /home/runner/work/astro/astro/packages/db
> astro-scripts build "src/**/*.ts" && tsc && pnpm types:virtual


> @astrojs/db@0.11.5 types:virtual /home/runner/work/astro/astro/packages/db
> tsc -p ./tsconfig.virtual.json

::endgroup::
::group::@astrojs/web-vitals:build
cache miss, executing 3515c7ffb364a1ad

> @astrojs/web-vitals@0.2.1 build /home/runner/work/astro/astro/packages/integrations/web-vitals
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::

 Tasks:    27 successful, 27 total
Cached:    6 cached, 27 total
  Time:    51.023s