Skip to content

test: preserve static params during Cache Components migration - #98209

Merged
aurorascharff merged 2 commits into
canaryfrom
codex/cache-components-empty-gsp-eval
Sep 3, 2026
Merged

test: preserve static params during Cache Components migration#98209
aurorascharff merged 2 commits into
canaryfrom
codex/cache-components-empty-gsp-eval

Conversation

@aurorascharff

@aurorascharff aurorascharff commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an agent eval covering migration of the legacy on-demand ISR pattern: force-static, revalidate, and an empty generateStaticParams result.

The assertions require the agent to enable Cache Components, preserve the generateStaticParams export, and replace its empty result with at least one parameter instead of deleting the function. The fixture also runs the Cache Components adoption skill with the established 30-minute timeout.

Updates the Cache Components migration guide to link each legacy route segment config to its migration instructions. The generateStaticParams section now explains that removing the export opts a dynamic route out of ISR and links to the ISR with Cache Components guide.

Eval results

Before the guide change

Treatment Result Duration Behavior
Baseline Passed (1/1) 482.9s Preserved generateStaticParams and returned a real seed.
Bundled docs / AGENTS.md Failed (0/1) 444.7s Deleted generateStaticParams, incorrectly treating omission as equivalent to returning an empty array.
Cache Components skill Passed (1/1) 527.6s Preserved generateStaticParams and returned a real seed.

The failed treatment was classified as a model failure. It read the bundled migration guide but concluded that deleting generateStaticParams preserved the defer-all-paths-to-runtime behavior. The eval catches that loss of on-demand ISR semantics.

The full run used the identical fixture immediately before its numeric identifier was changed from agent-044 to the final agent-054 name. The fixture and assertions were unchanged.

After the guide change

Treatment Result Duration Behavior
Baseline Passed (1/1) 739.5s Preserved generateStaticParams, returned FEATURED_EVENT_SLUG, and compared the migrated route with the original behavior.
Bundled docs / AGENTS.md Passed (1/1) 393.2s Read the revised migration and ISR guides, preserved generateStaticParams, and returned FEATURED_EVENT_SLUG.
Cache Components skill Passed (1/1) 547.3s Preserved generateStaticParams, returned FEATURED_EVENT_SLUG, and verified the build and representative routes in development.

In the first controlled pair, the bundled-docs treatment changed from failing to passing. Its final migration kept the route eligible for ISR, preserved the 60-second revalidation behavior with use cache and cacheLife, and produced a passing build.

Additional variance check

The bundled-docs treatment was run once more against each guide version:

Guide version First sample Second sample Observed result
Before this change Failed (444.7s) Passed (455.6s) 1/2 passed
After this change Passed (393.2s) Passed (425.2s) 2/2 passed

The second pre-change sample shows that the previous wording could still lead the agent to the correct migration. The revised wording passed both observed samples, suggesting that the explicit ISR consequence improves reliability. Two samples per version are not enough to claim a stable pass rate.

Verification

  • pnpm --filter=next build
  • pnpm eval agent-054-cache-components-empty-static-params --dry
  • pnpm eval agent-054-cache-components-empty-static-params

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Tests Passed

Commit: a83ffe6

@aurorascharff
aurorascharff marked this pull request as ready for review September 3, 2026 08:39
@aurorascharff
aurorascharff merged commit 87a1c18 into canary Sep 3, 2026
66 checks passed
@aurorascharff
aurorascharff deleted the codex/cache-components-empty-gsp-eval branch September 3, 2026 09:57
eps1lon pushed a commit that referenced this pull request Sep 7, 2026
- docs: explain origin matching for allowedOrigins and allowedDevOrigins
(#97805)
- docs: improve discovery summaries (#97982)
- docs(skills): preserve prefetched UI during Partial Prefetching
adoption (#97712)
- docs: add layout stability guidance for videos and iframes (#98070)
- docs: document preloading with Cache Components (#97864)
- docs: note that experimental.taint taints process.env (#98145)
- docs: add API reference pages for skipProxyUrlNormalize and
skipTrailingSlashRedirect (#97967)
- docs: keep generateStaticParams during Cache Components migration
(docs part of #98209)
- docs: Remove version label from Turbopack in Pages Router (#98316)

---------

Co-authored-by: Rich Haines <hello@richardhaines.dev>
Co-authored-by: Aurora Scharff <66901228+aurorascharff@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants