Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v20.2.0 release proposal #48020

Merged
merged 104 commits into from
May 16, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
d25c785
test: allow SIGBUS in signal-handler abort test
targos May 5, 2023
34bfb69
test: migrate message tests to use assertSnapshot
MoLow Apr 27, 2023
7323902
tools: fix jsdoc lint
MoLow Apr 30, 2023
36f7cfa
tools: update eslint to 8.39.0
nodejs-github-bot Apr 30, 2023
c923367
test: fix WPT state when process exits but workers are still running
panva May 3, 2023
f710676
doc: update BUILDING.md previous versions links
tniessen May 3, 2023
811b43c
doc,test: update the v8.startupSnapshot doc and test the example
joyeecheung May 3, 2023
2952cc5
src: add per-isolate SetFastMethod and Set[Fast]MethodNoSideEffect
joyeecheung Apr 28, 2023
d6d12bf
bootstrap: log isolate data info in mksnapshot debug logs
joyeecheung Apr 28, 2023
a0da234
fs: move fs_use_promises_symbol to per-isolate symbols
joyeecheung Apr 28, 2023
456fca0
bootstrap: initialize per-isolate properties of bindings separately
joyeecheung Apr 28, 2023
b04d51a
src: prefer data accessor of string and vector
VoltrexKeyva May 3, 2023
c092df9
doc: add ovflowd to collaborators
ovflowd May 3, 2023
977fd7c
meta: bump codecov/codecov-action from 3.1.1 to 3.1.3
dependabot[bot] May 3, 2023
9f06eac
meta: bump github/codeql-action from 2.2.9 to 2.3.2
dependabot[bot] May 3, 2023
e53e823
doc: replace EOL versions in README
tniessen May 3, 2023
17945a2
test: migrate a pseudo_tty test to use assertSnapshot
MoLow May 4, 2023
1666a14
doc: add valgrind suppression details
KevinEady May 4, 2023
6f3876c
worker: use snapshot in workers spawned by workers
joyeecheung May 4, 2023
7222f9d
path: indicate index of wrong resolve() parameter
sosoba May 4, 2023
1a7fc18
sea: allow requiring core modules with the "node:" prefix
RaisinTen May 4, 2023
c4596b9
sea: add option to disable the experimental SEA warning
RaisinTen May 4, 2023
d358317
src: get binding data store directly from the realm
joyeecheung Apr 5, 2023
1625ae1
quic: address recent coverity warning
mhdawson May 4, 2023
4bc17fd
src: avoid strcmp() with Utf8Value
tniessen May 4, 2023
c421761
src: fix creating an ArrayBuffer from a Blob created with `openAsBlob`
daeyeon Apr 24, 2023
739113f
module: block requiring `test/reporters` without scheme
MoLow May 5, 2023
54607bf
test: reduce WPT concurrency
panva May 5, 2023
a4d6543
http2: improve nghttp2 error callback
tniessen May 5, 2023
f489c67
node-api: get Node API version used by addon
vmoroz Apr 7, 2023
30f4f35
test: fix output tests when path includes node version
MoLow May 5, 2023
f96fb2e
doc: swap Matteo with Rafael in the stewards
RafaelGSS May 5, 2023
89c70dc
doc: add stability experimental to pm
RafaelGSS May 6, 2023
d23b1af
deps: update ada to 2.3.1
nodejs-github-bot May 6, 2023
9a362aa
doc: update supported version of FreeBSD to 12.4
targos May 6, 2023
93f1aa2
doc: fix params names
Semigradsky May 6, 2023
e9c6ee7
crypto: remove default encoding from pbkdf2
tniessen May 7, 2023
515c9b8
src: clarify the parameter name in `Permission::Apply`
daeyeon May 7, 2023
4dfc389
doc: improve `permission.has` description
daeyeon May 7, 2023
7b2f17c
deps: upgrade npm to 9.6.6
npm-cli-bot May 7, 2023
7e34f77
test: fix webcrypto wrap unwrap tests
panva May 7, 2023
09fb74a
crypto: fix webcrypto private/secret import with empty usages
panva May 7, 2023
8f547af
test: use appropriate usages for a negative import test
panva May 7, 2023
652b06d
meta: remove extra space in scorecard workflow
Mesteery May 1, 2023
0f58e48
meta: bump actions/checkout from 3.3.0 to 3.5.2
dependabot[bot] May 1, 2023
f7a8094
meta: bump actions/setup-python from 4.5.0 to 4.6.0
dependabot[bot] May 7, 2023
6c158e8
meta: bump step-security/harden-runner from 2.2.1 to 2.3.1
dependabot[bot] May 7, 2023
cad42e7
deps: V8: cherry-pick 1b471b796022
luyahan May 7, 2023
a11507e
src: stop copying code cache
kvakil May 7, 2023
1b06c1e
url: improve `isURL` detection
anonrig May 7, 2023
e457d89
buffer: combine checking range of sourceStart in `buf.copy`
deokjinkim May 7, 2023
72340c9
dgram: convert macro to template
tniessen May 8, 2023
9bc5d78
src: register ext reference for Fingerprint512
tniessen May 8, 2023
00668fc
child_process: use signal.reason in child process abort
debadree25 May 8, 2023
c19385c
module: refactor to use `normalizeRequirableId` in the CJS module loader
RaisinTen May 8, 2023
e22c686
tools: update eslint to 8.40.0
nodejs-github-bot May 9, 2023
4293cc4
src: support V8 experimental shared values in messaging
syg May 9, 2023
bf39f2d
doc: ntfs junction points must link to directories
bnoordhuis May 9, 2023
706c305
tools: update lint-md-dependencies to rollup@3.21.5
nodejs-github-bot May 9, 2023
c2b14b4
deps: update ada to 2.4.0
nodejs-github-bot May 9, 2023
7ed99e8
doc: mark global object as legacy
mertcanaltin May 9, 2023
f6ff318
tools: automate icu-small update
marco-ippolito May 9, 2023
c9ffc55
doc: document make lint-md-clean
mcollina May 10, 2023
106dc61
fs: make readdir recursive algorithm iterative
May 10, 2023
190596c
src: register external references for source code
kvakil May 10, 2023
99f8fca
deps: V8: cherry-pick a8a11a87cb72
targos Apr 18, 2023
b16f6da
deps: V8: cherry-pick 5f025d1ca2ca
targos Apr 18, 2023
bd553e7
src: rename SKIP_CHECK_SIZE to SKIP_CHECK_STRLEN
tniessen May 10, 2023
d81c54e
test_runner: omit inaccessible files from coverage
cjihrig May 10, 2023
9be9228
dns: call `ada::idna::to_ascii` directly from c++
anonrig May 10, 2023
aa2c7e0
test,crypto: update WebCryptoAPI WPT
panva May 10, 2023
42db1d5
test_runner: fix ordering of test hooks
philnash May 11, 2023
0c06bfd
src: move BlobSerializerDeserializer to a separate header file
RaisinTen May 9, 2023
af86625
permission: resolve reference to absolute path only for fs permission
daeyeon May 11, 2023
2a3d6d9
meta: add security-wg ping to permission.js
RafaelGSS May 11, 2023
6f5ba92
doc: remove broken link
Trott May 11, 2023
d799347
crypto: remove default encoding from scrypt
tniessen May 12, 2023
6e01958
test: unskip negative-settimeout.any.js WPT
panva May 12, 2023
a4fed6c
lib: update comment
sinkhaha May 5, 2023
47fea13
worker: support more cases when (de)serializing errors
MoLow May 12, 2023
b9771c9
doc: fix broken link
Trott May 12, 2023
bb33c74
test: add getRandomValues return length
MrJithil May 13, 2023
7ecc674
doc: fix broken link to TC39 import attributes proposal
Trott May 13, 2023
e6685f9
vm,lib: refactor microtaskQueue assignment logic
XadillaX Apr 28, 2023
4197a9a
http: prevent writing to the body when not allowed by HTTP spec
gerrard00 May 13, 2023
3727964
doc: update measure memory rejection information
yashLadha May 13, 2023
c05c0a2
meta: bump actions/setup-python from 4.5.0 to 4.6.0
Trott May 13, 2023
13118a1
doc: update description of global
tniessen May 13, 2023
1ec640a
esm: do not use `'beforeExit'` on the main thread
aduh95 May 14, 2023
1aec718
src: add cjs_module_lexer_version base64_version
MrJithil May 14, 2023
a0634d7
url: add value argument to has and delete methods
sankalp1999 May 14, 2023
a4e261e
tools: debug log for nghttp3
marco-ippolito May 14, 2023
f5b4b6d
meta: bump github/codeql-action from 2.3.2 to 2.3.3
Trott May 14, 2023
fd8bec7
meta: bump step-security/harden-runner from 2.3.1 to 2.4.0
Trott May 14, 2023
4a1e971
doc: add missing deprecated blocks to cluster
tniessen May 14, 2023
b7f13a8
deps: update simdutf to 3.2.9
nodejs-github-bot May 14, 2023
060c1d5
src: stop copying code cache, part 2
kvakil May 15, 2023
17befe0
test_runner: add shorthands to `test`
atlowChemi May 15, 2023
2bd869d
vm: fix crash when setting __proto__ on context's globalThis
F3n67u May 15, 2023
010d2ec
test: mark test-esm-loader-http-imports as flaky
tniessen May 15, 2023
786a1c5
src: deduplicate X509Certificate::Fingerprint*
tniessen May 15, 2023
7dd32f1
permission: remove unused function declaration
deokjinkim May 15, 2023
da27542
test_runner: use v8.serialize instead of TAP
MoLow May 15, 2023
258e9e7
2023-05-16, Version 20.2.0 (Current)
targos May 15, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
doc,test: update the v8.startupSnapshot doc and test the example
The API is now available to user-land run-time snapshots. So update
the example. This also makes the intention of the examples a bit
clearer and test it in our test suite.

PR-URL: #47468
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
  • Loading branch information
joyeecheung authored and targos committed May 12, 2023
commit 811b43c215beedf705508c8c3e32355aaf957939
85 changes: 53 additions & 32 deletions doc/api/v8.md
Original file line number Diff line number Diff line change
Expand Up @@ -921,15 +921,12 @@ added:
> Stability: 1 - Experimental

The `v8.startupSnapshot` interface can be used to add serialization and
deserialization hooks for custom startup snapshots. Currently the startup
snapshots can only be built into the Node.js binary from source.
deserialization hooks for custom startup snapshots.

```console
$ cd /path/to/node
$ ./configure --node-snapshot-main=entry.js
$ make node
# This binary contains the result of the execution of entry.js
$ out/Release/node
$ node --snapshot-blob snapshot.blob --build-snapshot entry.js
# This launches a process with the snapshot
$ node --snapshot-blob snapshot.blob
```

In the example above, `entry.js` can use methods from the `v8.startupSnapshot`
Expand All @@ -946,42 +943,66 @@ const zlib = require('node:zlib');
const path = require('node:path');
const assert = require('node:assert');

const {
isBuildingSnapshot,
addSerializeCallback,
addDeserializeCallback,
setDeserializeMainFunction,
} = require('node:v8').startupSnapshot;
const v8 = require('node:v8');

const filePath = path.resolve(__dirname, '../x1024.txt');
const storage = {};
class BookShelf {
storage = new Map();

assert(isBuildingSnapshot());
// Reading a series of files from directory and store them into storage.
constructor(directory, books) {
for (const book of books) {
this.storage.set(book, fs.readFileSync(path.join(directory, book)));
}
}

addSerializeCallback(({ filePath }) => {
storage[filePath] = zlib.gzipSync(fs.readFileSync(filePath));
}, { filePath });
static compressAll(shelf) {
for (const [ book, content ] of shelf.storage) {
shelf.storage.set(book, zlib.gzipSync(content));
}
}

addDeserializeCallback(({ filePath }) => {
storage[filePath] = zlib.gunzipSync(storage[filePath]);
}, { filePath });
static decompressAll(shelf) {
for (const [ book, content ] of shelf.storage) {
shelf.storage.set(book, zlib.gunzipSync(content));
}
}
}

setDeserializeMainFunction(({ filePath }) => {
console.log(storage[filePath].toString());
}, { filePath });
// __dirname here is where the snapshot script is placed
// during snapshot building time.
const shelf = new BookShelf(__dirname, [
'book1.en_US.txt',
'book1.es_ES.txt',
'book2.zh_CN.txt',
]);

assert(v8.startupSnapshot.isBuildingSnapshot());
// On snapshot serialization, compress the books to reduce size.
v8.startupSnapshot.addSerializeCallback(BookShelf.compressAll, shelf);
// On snapshot deserialization, decompress the books.
v8.startupSnapshot.addDeserializeCallback(BookShelf.decompressAll, shelf);
v8.startupSnapshot.setDeserializeMainFunction((shelf) => {
// process.env and process.argv are refreshed during snapshot
// deserialization.
const lang = process.env.BOOK_LANG || 'en_US';
const book = process.argv[1];
const name = `${book}.${lang}.txt`;
console.log(shelf.storage.get(name));
}, shelf);
```

The resulted binary will simply print the data deserialized from the snapshot
during start up:
The resulted binary will get print the data deserialized from the snapshot
during start up, using the refreshed `process.env` and `process.argv` of
the launched process:

```console
$ out/Release/node
# Prints content of ./test/fixtures/x1024.txt
$ BOOK_LANG=es_ES node --snapshot-blob snapshot.blob book1
# Prints content of book1.es_ES.txt deserialized from the snapshot.
```

Currently the API is only available to a Node.js instance launched from the
default snapshot, that is, the application deserialized from a user-land
snapshot cannot use these APIs again.
Currently the application deserialized from a user-land snapshot cannot
be snapshotted again, so these APIs are only available to applications
that are not deserialized from a user-land snapshot.

### `v8.startupSnapshot.addSerializeCallback(callback[, data])`

Expand Down
86 changes: 55 additions & 31 deletions test/fixtures/snapshot/v8-startup-snapshot-api.js
Original file line number Diff line number Diff line change
@@ -1,36 +1,60 @@
'use strict';

const fs = require('fs');
const zlib = require('zlib');
const path = require('path');
const assert = require('assert');

const {
isBuildingSnapshot,
addSerializeCallback,
addDeserializeCallback,
setDeserializeMainFunction
} = require('v8').startupSnapshot;

const filePath = path.resolve(__dirname, '../x1024.txt');
const storage = {};

assert(isBuildingSnapshot());

addSerializeCallback(({ filePath }) => {
console.error('serializing', filePath);
storage[filePath] = zlib.gzipSync(fs.readFileSync(filePath));
}, { filePath });

addDeserializeCallback(({ filePath }) => {
console.error('deserializing', filePath);
storage[filePath] = zlib.gunzipSync(storage[filePath]);
}, { filePath });

setDeserializeMainFunction(({ filePath }) => {
console.log(storage[filePath].toString());
}, { filePath });
assert.throws(() => setDeserializeMainFunction(() => {
const fs = require('node:fs');
const zlib = require('node:zlib');
const path = require('node:path');
const assert = require('node:assert');

const v8 = require('node:v8');

class BookShelf {
storage = new Map();

// Reading a series of files from directory and store them into storage.
constructor(directory, books) {
for (const book of books) {
this.storage.set(book, fs.readFileSync(path.join(directory, book)));
};
}

static compressAll(shelf) {
for (const [ book, content ] of shelf.storage) {
shelf.storage.set(book, zlib.gzipSync(content));
}
}

static decompressAll(shelf) {
for (const [ book, content ] of shelf.storage) {
shelf.storage.set(book, zlib.gunzipSync(content));
}
}
}

// __dirname here is where the snapshot script is placed
// during snapshot building time.
const shelf = new BookShelf(__dirname, [
'book1.en_US.txt',
'book1.es_ES.txt',
'book2.zh_CN.txt',
]);

assert(v8.startupSnapshot.isBuildingSnapshot());

// On snapshot serialization, compress the books to reduce size.
v8.startupSnapshot.addSerializeCallback(BookShelf.compressAll, shelf);
// On snapshot deserialization, decompress the books.
v8.startupSnapshot.addDeserializeCallback(BookShelf.decompressAll, shelf);
v8.startupSnapshot.setDeserializeMainFunction((shelf) => {
// process.env and process.argv are refreshed during snapshot
// deserialization.
const lang = process.env.BOOK_LANG || 'en_US';
const book = process.argv[1];
const name = `${book}.${lang}.txt`;
console.error('Reading', name);
console.log(shelf.storage.get(name).toString());
}, shelf);

assert.throws(() => v8.startupSnapshot.setDeserializeMainFunction(() => {
assert.fail('unreachable duplicated main function');
}), {
code: 'ERR_DUPLICATE_STARTUP_SNAPSHOT_MAIN_FUNCTION',
Expand Down
20 changes: 16 additions & 4 deletions test/parallel/test-snapshot-api.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
'use strict';

// This tests snapshot JS API
// This tests snapshot JS API using the example in the docs.

require('../common');
const assert = require('assert');
Expand All @@ -20,11 +20,20 @@ tmpdir.refresh();
const blobPath = path.join(tmpdir.path, 'snapshot.blob');
const entry = fixtures.path('snapshot', 'v8-startup-snapshot-api.js');
{
for (const book of [
'book1.en_US.txt',
'book1.es_ES.txt',
'book2.zh_CN.txt',
]) {
const content = `This is ${book}`;
fs.writeFileSync(path.join(tmpdir.path, book), content, 'utf8');
}
fs.copyFileSync(entry, path.join(tmpdir.path, 'entry.js'));
const child = spawnSync(process.execPath, [
'--snapshot-blob',
blobPath,
'--build-snapshot',
entry,
'entry.js',
], {
cwd: tmpdir.path
});
Expand All @@ -41,15 +50,18 @@ const entry = fixtures.path('snapshot', 'v8-startup-snapshot-api.js');
const child = spawnSync(process.execPath, [
'--snapshot-blob',
blobPath,
'book1',
], {
cwd: tmpdir.path,
env: {
...process.env,
BOOK_LANG: 'en_US',
}
});

const stdout = child.stdout.toString().trim();
const file = fs.readFileSync(fixtures.path('x1024.txt'), 'utf8');
assert.strictEqual(stdout, file);
const stderr = child.stderr.toString().trim();
assert.strictEqual(stderr, 'Reading book1.en_US.txt');
assert.strictEqual(stdout, 'This is book1.en_US.txt');
assert.strictEqual(child.status, 0);
}