Repository navigation
Use Harper readonly mode for Next.js builds - #41
Conversation
kriszyp
left a comment
There was a problem hiding this comment.
I believe we need to (somehow) flush all the databases so the new process can "see" all the data.
|
No, I think we will need to add one. |
|
|
||
| // Next.js generates static pages using child processes and only a single process can open the | ||
| // RocksDB databases, so force Harper to start in read-only mode | ||
| process.env.READONLY = 'true'; |
There was a problem hiding this comment.
I'm surprised this wasn't prefixed with like ROCKSDB_ or HARPER_, but thats an upstream thing so no concern here.
|
looks like this needs to wait until Harper is updated (or |
|
Yeah, this PR needs to wait a little bit. |
|
I will convert to draft. Feel free to click the "Ready for Review" button when its good to go again. |
|
We're just waiting on Harper 5.1.0 to ship. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
|
@cb1kenobi think this is ready for a rebase? Do we have the right stuff published where this can happen? |
|
@dawsontoth Updated the branch. Dropped Node.js 20 from test matrix, added Node.js 26. Had to update some dependencies to get the tests to run. |
dawsontoth
left a comment
There was a problem hiding this comment.
Nice updates, though!
|
@cb1kenobi if we get the GHA for release in before this one, then we can rest real releases. |
|
Hmm, tests are failing due to "Error: ENOSPC: no space left on device". That can't be good. |
dawsontoth
left a comment
There was a problem hiding this comment.
can bAIrber take a look at the failing integration tests?
Next.js uses child processes to generate static pages. When those pages reference Harper data, the child process tries to open the RocksDB databases and fails because only one process can hold the write lock at a time. Set HARPER_READONLY=true before Next.js spawns its build workers so they open Harper in read-only mode. Call flushDatabases() first so the child processes can see all committed data. Also: drop Node.js 20 from the CI test matrix, add Node.js 26; update dependencies; pass --install-links to npm install in install-fixtures.js to prevent circular symlinks (ELOOP / ENOSPC) caused by the file: protocol pointing back to the project root. Co-authored-by: Cursor <cursoragent@cursor.com>
@harperfast/nextjs 2.2.2 fixed the multi-thread build race (serializes the build across worker threads — HarperFast/nextjs#52), which is the only reason these templates shipped `prebuilt: true` + build-first scripts. Revert to the same server-side-build model as the other templates: - config.yaml: drop `prebuilt: true` (Harper builds on `harper run`/deploy). - scripts: `dev` → `harper dev .`, `start` → `harper run .`, `deploy` → `harper deploy_component ...` (no `next build &&` prefix). - bump `@harperfast/nextjs` to `^2.2.3` so the build-serialization fix is required. - README deploy note back to "Harper builds on the server, no local build". Verified on Harper 5.1.22 with @harperfast/nextjs 2.2.3: a fresh scaffold builds and serves multi-threaded via `harper run` (no prebuilt, no race) and the counter works. (Server code still uses the injected `tables`/`transaction` globals, not `import 'harper'` — that stays until HarperFast/nextjs#41 + #51 land.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kriszyp
left a comment
There was a problem hiding this comment.
I think this would still be great to update and merge.
… in readonly mode HARPER_READONLY was set on process.env before calling next build so that Next.js child processes would open the Harper databases in read-only mode, avoiding the RocksDB lock conflict. However, it was never unset, so the server process itself also ran in read-only mode after the build completed, causing all Harper data queries to return nothing and pages to render blank. Save and restore the previous value in the finally block, matching the same pattern used for --expose-internals / process.execArgv. Co-authored-by: Cursor <cursoragent@cursor.com>
@harperfast/nextjs 2.2.2 fixed the multi-thread build race (serializes the build across worker threads — HarperFast/nextjs#52), which is the only reason these templates shipped `prebuilt: true` + build-first scripts. Revert to the same server-side-build model as the other templates: - config.yaml: drop `prebuilt: true` (Harper builds on `harper run`/deploy). - scripts: `dev` → `harper dev .`, `start` → `harper run .`, `deploy` → `harper deploy_component ...` (no `next build &&` prefix). - bump `@harperfast/nextjs` to `^2.2.3` so the build-serialization fix is required. - README deploy note back to "Harper builds on the server, no local build". Verified on Harper 5.1.22 with @harperfast/nextjs 2.2.3: a fresh scaffold builds and serves multi-threaded via `harper run` (no prebuilt, no race) and the counter works. (Server code still uses the injected `tables`/`transaction` globals, not `import 'harper'` — that stays until HarperFast/nextjs#41 + #51 land.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
The wait is over but the blocker moved — #60 targets this branch with the fix, plus the regression test this was missing. @kriszyp @Ethan-Arrowood on "we'll need to add one" / "wait until Harper is updated": Harper already has What's failing is that components don't see the package's exports. Harper hands them a So
The flush does work today — under Two follow-ups worth deciding:
Also fixed in #60: 🤖 Generated by Claude Code (Opus 5), reviewed by @dawsontoth |
…er it with a test (#60) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…per data HARPER_READONLY and flushDatabases do not exist in Harper v5.0.x, so static generation that reads Harper data at build time requires v5.1+. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
🎉 This PR is included in version 2.2.5 🎉 The release is available on: Your semantic-release bot 📦🚀 |
…enerator (#71) * ci: lint the PR title, and match the analyzer's preset to the notes generator Squash is the only merge method here, and `squash_merge_commit_title` is COMMIT_OR_PR_TITLE, so for any PR with more than one commit the PR title becomes the subject of the commit that lands on main — and that subject is the only thing semantic-release reads. commitlint sees the commits inside the PR, which the squash then discards, so a title with no conventional type is a silent no-release. #41 shipped inside 2.2.5 with no note of its own, and #65 produced no release at all. A title check now runs on every pull request, including on `edited`, so a retitle re-checks. `@semantic-release/commit-analyzer` also carried no `preset` and so defaulted to `angular`, while the notes generator was explicitly `conventionalcommits`. The two now match. That is not only tidiness: `angular` does not recognise the `!` breaking-change shorthand, so `feat!: …` passed commitlint and then analysed as no release at all. Verified against both presets — `feat!:` is `none` under angular and `major` under conventionalcommits; every other type agrees. CONTRIBUTING gains the rules and a type-to-release table. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: record how squashing treats the two breaking-change forms The `!` is read only from the commit subject and a `BREAKING CHANGE:` footer only from the body, so under squash merging the bang counts in the PR title while the footer counts in a commit inside the PR. Both verified against the configured analyzer, including the shape GitHub actually produces. Also rewraps the section: the file keeps a paragraph on one line, and the first draft hard-wrapped at 100 columns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: list every commit type the release config declares The no-release row named four of the seven. `style`, `build` and `ci` are declared in .releaserc.json too, and all three analyse as no release. Also notes that "none" is about triggering a release, not visibility: every declared type has its own section, so it still appears in the notes of whatever release it rides along in. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: drop the style and build note sections this repo does not use Both came in with the release config and neither is used here. Removing them from the presetConfig changes nothing about release types — both still analyse as no release — but such a commit is now hidden from the notes rather than given its own section. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
## [2.3.0](v2.2.5...v2.3.0) (2026-10-08) ### Features * back the `use cache` directive with Harper and persist per-entry cache lives ([#65](#65)) ([5bcd077](5bcd077)) ### Continuous Integration * lint the PR title, and match the analyzer's preset to the notes generator ([#71](#71)) ([9ebd026](9ebd026)), closes [#41](#41) [#65](#65)
Next.js uses child processes to generate static pages. When those pages reference Harper data, it will try to load those databases and error with:
Harper has a new
READONLYenvironment variable that will allow a RocksDB database to be opened by another process in read-only mode allowing the static pages to be generated.