Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] canary from vercel:canary #568

Open
wants to merge 5,718 commits into
base: canary
Choose a base branch
from
Open

Conversation

pull[bot]
Copy link

@pull pull bot commented Feb 8, 2024

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull bot requested a review from timneutkens as a code owner February 8, 2024 01:29
@pull pull bot added the ⤵️ pull label Feb 8, 2024
huozhi and others added 28 commits October 7, 2024 17:57
prerenderStore was unified with workUnitStore before I landed random
extension. This extends the refactor to the newly landed code
…t unmounting and losing state (#70397)

### What?

Update 04-linking-and-navigating.mdx, 4. Partial Rendering section, add
information that rerenders of sibling routes mean full rerender of the
page, unmounting and losing the state of the page

### Why?
I've spent a lot of time trying to understand why in the app router when
switching between sibling routes (and Dynamic Segments) I lose scroll
position. Finally, find a discussion that this is the intentional
behaviour of the Next.js team. Would be good if the Next.js users could
find information in the documentation instead of searching for issues
and discussions in the documentation.

#50711

### How?
Updated docs

---------

Co-authored-by: JJ Kasper <jj@jjsweb.site>
Closes PACK-3288
Closes #70424

Sentry uses `open-telemetry`, which uses `require-in-the-middle` which:
- overrides the global `Module.prototype.require`, even when bundled
- reads `require.cache` which when bundled is Turbopack's registry (not
the outer Node.js one)


Because of this, `require-in-the-middle` is broken/results in a broken
environment when it's bundled (e.g. via `instrumentation.js`).
We can't guarantee this port will always be available so use random one
instead

x-ref:
https://github.com/vercel/next.js/actions/runs/11221640914/job/31192432034
`TaskInput` isn't needed/used for a bare `ResolvedVc`, as we'll expose
`ResolvedVc` arguments as `Vc`, but it is useful for structs that
contain `ResolvedVc` and want to derive `TaskInput`.

This PR also ports `next_core::app_structure::Entrypoint` to use
`ResolvedVc` as a way of testing this.
These are better for telemetry and Date.now() will start to be dynamic
in prerenders
This:

- Updates the toolchain file
- Adds `#![feature(arbitrary_self_types_pointers)]` as necessary
- Removes opt-ins for now-stable features like
`option_get_or_insert_default`
- Removes now-detectable dead code
…70815)

Removes the uses of generic vc collections in
`next-core/src/app_structure.rs` and it's callsites.

## Why?

Rather than extended support for `Vc` generics to `ResolvedVc`, I plan
to remove support for them entirely. That means fixing all the callsites
(there's not many).

Removing this is needed to get us to a point where 100% of structs can
be `ResolvedValue`, because structs using these fields cannot otherwise
be ported over to `ResolvedVc`.

## Okay, but Why?

Explained here:
vercel/turborepo#8843 (comment)

I expect removing support for this will decrease the overall size of the
codebase, as the logic for supporting generics is rather complicated.
stacked on #70924 

Incremental cache tracks time using Date.now() and new Date(). this
change updates these timers to use perf_hooks or performance. These are
not susceptible to global clock changes and will not be considered
dynamic for purposes of
dynamicIO in the future
)

`default` is from `inquirier`, for prompts, it's `initial`.

x-ref: https://www.npmjs.com/package/prompts

This made directory value undefined when user `Enter`ed the default value.

Before:

![CleanShot 2024-10-08 at 05 39 17](https://github.com/user-attachments/assets/1e10abe0-0138-4505-a694-2fb3278258a4)

After:

![CleanShot 2024-10-08 at 05 39 48](https://github.com/user-attachments/assets/bf24d708-4458-4396-8115-d467d100571f)
…rics in non-test code (#70816)

After this change, `cargo check` on a branch that deletes vc generics
support compiles successfully.

## Why?

Rather than extended support for `Vc` generics to `ResolvedVc`, I plan
to remove support for them entirely. That means fixing all the callsites
(there's not many).

Removing this is needed to get us to a point where 100% of structs can
be `ResolvedValue`, because structs using these fields cannot otherwise
be ported over to `ResolvedVc`.

## Okay, but Why?

Explained here:
vercel/turborepo#8843 (comment)

I expect removing support for this will decrease the overall size of the
codebase, as the logic for supporting generics is rather complicated.
…#70916)

Date can be considered IO when it infers the current time when using
`new Date()` or `Date.now()`. Both of these are queries to the computer
clock and while they are synchronous they are semantically IO. When
`dyanmicIO` is enabled these APIs should be treated like other IO and be
excluded from prerenders unless explicitly cached
crypto random bytes are IO semantically even if they are synchronous. We
instrument the sync crypto APIs that allow access to randomness to be be
dynamic while prerendering.
…when fail to get next version (#70929)

### Why?

Instead of validating workspace, just look for next version on cwd. If fails, give warn to run on the nextjs app dir if using monorepo.
…#70930)

Without `onCancel` handling, even if `CTRL + C`, the prompt will not
exit and continue to next step. Added it to missing prompts calls.
In Next 15, route handlers should no longer cache any `fetch` requests
by default. We do this by bailing out of caching if no explicit cache
config is defined. However, this misses the case where `cache:
"default"` is specified. Since the default caching behavior is to not
cache, this should also opt into the automatic no cache behavior.

This adds a test for the route handler and still validates that it
doesn't impact page-level ISR cache.
Lower a bit of certainty in stableness with Turbopack dev yet. Inform
user clearly to add `--turbo` manually if `next dev` wasn't found.
Stacked on #70806.

We want `after` callbacks to run in the same ALS context that `after`
was called in. This can be achieved via
https://nodejs.org/api/async_context.html#static-method-asynclocalstoragebindfn

We want this be cause these should be as close as possible in every way
(except timing):
```
after(() => x())
after(x())
await x()
```

We'll need a different approach for disabling setting cookies (and
revalidate). This will be done in a follow up.

---------

Co-authored-by: Janka Uryga <lolzatu2@gmail.com>
### What?
Consistently casing of pathname throughout the code-base. It is a simple
search and replace of all the ocuences 'athName' to 'athname'.

### Why?
I believe that the usage of the casing [pP]athName is likely a mistake,
as it is usually typed [pP]athname in the rest of the repo.
I think a case sensitive search replace from 'athName' to 'athname'
would be a small thing, and prevent this from growing.

### How?
I searched for pathname in the repo, and found a huge amount of hits.
Searching for 'athName' only returned a few.
The usePathname hook was the reason I came across this inconsistency. We
had the inconsistency in our own Next.js project, ie. `const pathName =
usePathname()`, and I noticed that when searching inside my project with
cases-sensitivity I found two hits of 'pathName' inside the
documentation for LoadableManifest.
This lead me to fork the repo and search, and then I found a few more
cases.
In each file I did a search for conflicts: if for example pathname was
already defined, it would be considered breaking if I changed a pathName
to pathname.
I could not find any cases where the variable was defined twice with the
same casing, so I believe this PR is safe.
I guess the test coverage/pipeline will tell us if this change is
breaking? If not I don't think I would be able to contribute that much
further in the time of writing.

Co-authored-by: JJ Kasper <jj@jjsweb.site>
### Why?

Previously, we were showing ALL the descriptions of the codemods which
made hard to read each lines of codemods.

### How?

Show description only of current highlighted option. Also, display the
version of the codemod.

### Before

`@next/codemod`


https://github.com/user-attachments/assets/2537c59e-c4a8-4cdd-af34-68cabf56354b

`@next/codemod upgrade`


https://github.com/user-attachments/assets/0fc05776-b398-4e13-b6ea-1e351fff2194

### After

`@next/codemod`


https://github.com/user-attachments/assets/553e5952-bd2e-4a19-99b0-e3fdab10d868

`@next/codemod upgrade`


https://github.com/user-attachments/assets/375c9abc-da3e-4c23-a03d-12def4c6d2a3

---------

Co-authored-by: Zack Tanner <1939140+ztanner@users.noreply.github.com>
Co-authored-by: Jiachi Liu <inbox@huozhi.im>
The previous PRs in this stack remove the only uses/callsites. This
removes support for vc generics!

There's still a small bit of support left in the form of
`VcValueType::Repr`, but that touches more things, so that'll be a
follow-up PR.

## Why?

Rather than extended support for `Vc` generics to `ResolvedVc`, I plan
to remove support for them entirely. That means fixing all the callsites
(there's not many).

Removing this is needed to get us to a point where 100% of structs can
be `ResolvedValue`, because structs using these fields cannot otherwise
be ported over to `ResolvedVc`.

## Okay, but Why?

Explained here:
vercel/turborepo#8843 (comment)

I expect removing support for this will decrease the overall size of the
codebase, as the logic for supporting generics is rather complicated.
### What?

Takes regular snapshots of the state of the system.

Stores task cache and task data in a lmdb database. 

Currently active operations are paused, stored and continued on restore.

Task Cache need large keys, so key hashing and lookup is added on top of
LMDB for that.
Noticed these were unused while trying to apply the `Vc` -> `ResolvedVc`
codemods.
### What?

add experimental option to enable persistent caching
devjiwonchoi and others added 30 commits October 15, 2024 19:42
#71258)

Same as #70897, but for the Edge runtime. The changes are based on what
we're already doing for app pages that are using the Edge runtime.
With the default debug console:

<img width="1075" alt="Screenshot 2024-10-15 at 13 22 12"
src="https://github.com/user-attachments/assets/5f87b1da-1ac1-4483-bf2e-4903b43b33d9">

With the integrated terminal:

<img width="1075" alt="Screenshot 2024-10-15 at 13 22 20"
src="https://github.com/user-attachments/assets/a7cb6b80-c028-46fe-9afc-5f4eceaf172a">
This re-adds the gate that verifies canary is being used before allowing
certain experimental features to be enabled. This is because these
features are undergoing frequent changes where we want to make sure that
the latest iterations are being validated.
Small follow-up to #71263 this
should be `Symbol.for` so we use the same reference instead creating a
new Symbol.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

### Improving Documentation

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

### Adding or Updating Examples

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md


## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

### What?

### Why?

### How?

Closes NEXT-
Fixes #

-->
Updates `unstable_after` docs to match changes from
#71231

---------

Co-authored-by: Delba de Oliveira <32464864+delbaoliveira@users.noreply.github.com>
Fixes an issue where using `after` in a `runtime = "edge"` page would
throw an error saying that `waitUntil` is missing. This was because we
weren't correctly passing `waitUntil` into the edge runtime sandbox.

(More precisely, the WebServer we run for those app-page/app-route
handlers wasn't receiving `waitUntil` -- middleware goes through a
different codepath and thus was fine)

This PR (ab)uses the `"@next/request-context"` global ALS (intended for
platforms to provide `waitUntil` to Next.js) to fix this -- it works
because the wrapped WebServer will read `"@next/request-context"` and
thus use the correct `waitUntil`.
- Rework bullet points from blog post
- Fix relative links mentioning Notion from copy/pasta
- Move the unstable APIs to the bottom of the list
Co-authored-by: eps1lon <sebastian.silbermann@vercel.com>
Minor grammar fixes.

---------

Co-authored-by: JJ Kasper <jj@jjsweb.site>
There's no artifact called `next-swc-binaries`, all the other places use
`next-swc-binaries-*`
The name's coming from here:
https://github.com/vercel/next.js/blob/bc309364d09c11cc6c149e643fa012c4d88dcb6c/.github/workflows/build_and_deploy.yml#L377

Example failing run:
https://github.com/vercel/next.js/actions/runs/11352776677/job/31576710878
It's failing because `download-artifact` didn't download anything
(because of the incorrect name), and the `native` directory no longer
exists (it used to have a .gitkeep inside, but we removed that a while
ago), so it just crashes
Adds the following missing pieces to Turbopack:
- apply server action transforms to route handler modules
- generate client reference manifests for route handler app endpoints
- make `experimental.cacheLife` config serialisable, to be injected into
the edge route handler modules
    - the `nextConfig` injection was already implemented in:
    #71258
```
cacheLife: {
  "seconds": {
    stale: 30, // 30 seconds
    revalidate: 1, // 1 second
    expire: 1, // 1 minute
  },
  "minutes": {
    stale: 60 * 5, // 5 minutes
    revalidate: 60, // 1 minute
    expire: 60 * 60, // 1 hour
  },
  "hours": {
    stale: 60 * 5, // 5 minutes
    revalidate: 60 * 60, // 1 hour
    expire: 60 * 60 * 24, // 1 day
  },
  "days": {
    stale: 60 * 5, // 5 minutes
    revalidate: 60 * 60 * 24, // 1 day
    expire: 60 * 60 * 24 * 7, // 1 week
  },
  "weeks": {
    stale: 60 * 5, // 5 minutes
    revalidate: 60 * 60 * 24 * 7, // 1 week
    expire: 60 * 60 * 24 * 30, // 1 month
  },
  "max": {
    stale: 60 * 5, // 5 minutes
    revalidate: 60 * 60 * 24 * 30, // 1 month
    expire: Infinity, // Unbounded. Whatever max your host stores.
  }
}
```

The stale time is either 30 seconds or 5 minutes which correspond to our
existing default for dynamic/static data. If you have overridden the
`staleTimes` options, we default for those. `"seconds"` means basically
dynamic.

The revalidate time is the beginning of the range and the expire is the
end of that range. So `"days"` refreshes somewhere after 1 day and
before 1 week.
This is basically acting like when you'd have Syncpack installed.
Some of these packages probably don't make sense to
have installed but ~this is america~ why be opinionated
about it when we only bump the dependency is already
declared.

## Test plan

- [x] updates `eslint-config-next` in https://github.com/rauchg/blog:
`node ~/packages/next-codemod/bin/next-codemod.js upgrade canary`

Co-authored-by: JJ Kasper <jj@jjsweb.site>
We need to encode FormData when we generate a complex cache key since
that's what RSC uses for encodeReply.

However, using the standard `multipart/form-data` encoding in `new
Response().arrayBuffer()` uses a large delimiter that's inefficient but
that delimiter is also randomly generated which completely defeats the
purpose of using it as a cache key.

Instead, we can use the size of each part as the delimiter.

It is important to something that is not just a character because
otherwise it's a potential security hole since someone could carefully
tailor input to cause cache poisoning.

E.g. if we used `:` as the delimiter then `[['a', 'b'],['c', 'd']]`
would be encoded as `a:b:c:d` but so would `[['a', 'b:c:d']]`.
To seed caches in dev mode we implemented a prefetch if one had not been
done in the recent past. The implementation isn't quite working b/c it
doesn't match the prefetch header properly but the mechanics are there.
One problem however is the prefetch streams so we end up starting the
regular render before the prefetch render is complete.

The ideal cache warmup is to only render until there are no more caches
to fill in the prefetch. Additionally we can disable certain things like
dynamic Request apis and fetches so they stall forever too. This
suggests that there is enough of a difference in the needs of the
prefetch for cache warming that we ought to implement it as it's own
request type. This change implements the mechanics of triggering the
warmup prefetch but does not yet implement the changes to things like
the request or fetch apis behavior. I will follow up with further
changes on top of this commit.
…nested directives (#71332)

This adds support of collecting and binding encrypted closure variables
to references of `"use cache"` functions. These closures can be mixed
with `"use server"` in any order, too.

Check the tests for more information.
…e directive transform (#71334)

This PR fixes the transform of handling `export default async function()
{}` and `export const foo = async function() {}` cases (both are
exported `fnExpr`, instead of `fnDecl`) in a `"use cache"` annotated
file.
Reimplements the logic of determining if we are dynamic by reading the
AbortController's signal when handling errors. This is more robust than
checking error identity because it's not guaranteed that the error we
receive is literally the instance that was passed to the abort call.
In #71231 we've added `onAfterTaskError`, which is intended for handling
errors that happened within `unstable_after` (currently only used to
fail the build if anything is thrown during prerendering). It was being
called correctly for the callback form, `unstable_after(() => { ... })`,
but we missed the fact that `unstable_after(promise)` is also valid, and
weren't calling it for rejected promises.

(Apparently we also weren't printing any kind of error message for
those, and it's been that way since the start. my bad.)

This PR unifies the error handling for callbacks and promises.
This stabilizes Partial Prerendering Fallbacks (PFPR) so that when
Partial Prerendering (PPR) is enabled, so is PFPR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.