Skip to content

Commit d57ec48

Browse files
committed
module: add a read-only mode to the compile cache
A compile cache generated ahead of time and shipped inside an application package should only ever be read: the package may be immutable or covered by an integrity check, and a cache directory that appears at run time would be a surprise. Add readOnly to module.enableCompileCache() and NODE_COMPILE_CACHE_READONLY=1: existing entries are loaded as before, nothing is serialized or persisted, flushCompileCache() is a no-op, and the cache directory is used as found rather than created, so enabling against a missing directory fails instead of making one. Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
1 parent bc813a7 commit d57ec48

9 files changed

Lines changed: 189 additions & 26 deletions

File tree

‎doc/api/cli.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3729,6 +3729,15 @@ Enable the [module compile cache][] for the Node.js instance. See the documentat
37293729
When set to 1, the [module compile cache][] can be reused across different directory
37303730
locations as long as the module layout relative to the cache directory remains the same.
37313731

3732+
### `NODE_COMPILE_CACHE_READONLY=1`
3733+
3734+
<!-- YAML
3735+
added: REPLACEME
3736+
-->
3737+
3738+
When set to 1, the [module compile cache][] only reads existing entries from
3739+
its directory: nothing is written to it and it is not created if missing.
3740+
37323741
### `NODE_DEBUG=module[,…]`
37333742

37343743
<!-- YAML

‎doc/api/module.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -421,6 +421,15 @@ There are two ways to enable the portable mode:
421421
422422
2. Setting the environment variable: [`NODE_COMPILE_CACHE_PORTABLE=1`][]
423423
424+
### Read-only compile cache
425+
426+
A cache that was generated ahead of time, for example at build time to be
427+
shipped inside an application package, can be enabled with `readOnly: true`
428+
(or [`NODE_COMPILE_CACHE_READONLY=1`][]). Node.js then loads whatever entries
429+
the directory holds and never writes to it: modules without a usable entry are
430+
compiled as usual but not persisted, [`module.flushCompileCache()`][] is a
431+
no-op, and the directory is not created if it is missing.
432+
424433
### Limitations of the compile cache
425434
426435
Currently when using the compile cache with [V8 JavaScript code coverage][], the
@@ -494,6 +503,9 @@ The following constants are returned as the `status` field in the object returne
494503
<!-- YAML
495504
added: v22.8.0
496505
changes:
506+
- version: REPLACEME
507+
pr-url: https://github.com/nodejs/node/pull/00000
508+
description: Add the `readOnly` option.
497509
- version:
498510
- v25.4.0
499511
- v24.15.0
@@ -520,6 +532,11 @@ changes:
520532
the cache can be reused even if the project directory is moved. This is a best-effort
521533
feature. If not specified, it will depend on whether the environment variable
522534
[`NODE_COMPILE_CACHE_PORTABLE=1`][] is set.
535+
* `readOnly` {boolean} Optional. If `true`, existing cache entries in `directory` are
536+
used but nothing is ever written to it, and the directory is not created when it does
537+
not exist (enabling then fails). Meant for caches generated ahead of time and shipped
538+
with an application. If not specified, it will depend on whether the environment
539+
variable [`NODE_COMPILE_CACHE_READONLY=1`][] is set.
523540
* Returns: {Object}
524541
* `status` {integer} One of the [`module.constants.compileCacheStatus`][]
525542
* `message` {string|undefined} If Node.js cannot enable the compile cache, this contains
@@ -2059,6 +2076,7 @@ returned object contains the following keys:
20592076
[`--require`]: cli.md#-r---require-module
20602077
[`NODE_COMPILE_CACHE=dir`]: cli.md#node_compile_cachedir
20612078
[`NODE_COMPILE_CACHE_PORTABLE=1`]: cli.md#node_compile_cache_portable1
2079+
[`NODE_COMPILE_CACHE_READONLY=1`]: cli.md#node_compile_cache_readonly1
20622080
[`NODE_DISABLE_COMPILE_CACHE=1`]: cli.md#node_disable_compile_cache1
20632081
[`NODE_V8_COVERAGE=dir`]: cli.md#node_v8_coveragedir
20642082
[`Object.freeze()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze

‎doc/node.1‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1862,6 +1862,10 @@ module compile cache for details.
18621862
When set to 1, the module compile cache can be reused across different directory
18631863
locations as long as the module layout relative to the cache directory remains the same.
18641864
.
1865+
.It Ev NODE_COMPILE_CACHE_READONLY Ar 1
1866+
When set to 1, the module compile cache only reads existing entries from
1867+
its directory: nothing is written to it and it is not created if missing.
1868+
.
18651869
.It Ev NODE_DEBUG Ar module[,…]
18661870
\fB','\fR-separated list of core modules that should print debug information.
18671871
.

‎lib/internal/modules/helpers.js‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -466,21 +466,25 @@ function stringify(body) {
466466
* after this method is called.
467467
* This method accepts either:
468468
* - A string: path to the cache directory.
469-
* - An options object `{directory?: string, portable?: boolean}`:
469+
* - An options object `{directory?: string, portable?: boolean, readOnly?: boolean}`:
470470
* - `directory`: A string path to the cache directory.
471471
* - `portable`: If `portable` is true, the cache directory will be considered relative.
472472
* Defaults to `NODE_COMPILE_CACHE_PORTABLE === '1'`.
473+
* - `readOnly`: If `readOnly` is true, existing cache entries are used but nothing is
474+
* written, and the cache directory is not created. Defaults to
475+
* `NODE_COMPILE_CACHE_READONLY === '1'`.
473476
* If cache directory is undefined, it defaults to the `NODE_COMPILE_CACHE` environment variable.
474477
* If `NODE_COMPILE_CACHE` isn't set, it defaults to `path.join(os.tmpdir(), 'node-compile-cache')`.
475-
* @param {string | { directory?: string, portable?: boolean } | undefined} options
478+
* @param {string | { directory?: string, portable?: boolean, readOnly?: boolean } | undefined} options
476479
* @returns {{status: number, message?: string, directory?: string}}
477480
*/
478481
function enableCompileCache(options) {
479482
let portable;
483+
let readOnly;
480484
let directory;
481485

482486
if (typeof options === 'object' && options !== null) {
483-
({ directory, portable } = options);
487+
({ directory, portable, readOnly } = options);
484488
} else {
485489
directory = options;
486490
}
@@ -490,7 +494,10 @@ function enableCompileCache(options) {
490494
if (portable === undefined) {
491495
portable = process.env.NODE_COMPILE_CACHE_PORTABLE === '1';
492496
}
493-
const nativeResult = _enableCompileCache(directory, portable);
497+
if (readOnly === undefined) {
498+
readOnly = process.env.NODE_COMPILE_CACHE_READONLY === '1';
499+
}
500+
const nativeResult = _enableCompileCache(directory, portable, readOnly);
494501
const result = { status: nativeResult[0] };
495502
if (nativeResult[1]) {
496503
result.message = nativeResult[1];

‎src/compile_cache.cc‎

Lines changed: 52 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -250,7 +250,8 @@ CompileCacheEntry* CompileCacheHandler::GetOrInsert(Local<String> code,
250250
// If the portable cache is enabled and it seems possible to compute the
251251
// relative position from an absolute path, we use the relative position
252252
// in the cache key.
253-
if (portable_ == EnableOption::PORTABLE && IsAbsoluteFilePath(file_path)) {
253+
if (HasOption(portable_, EnableOption::PORTABLE) &&
254+
IsAbsoluteFilePath(file_path)) {
254255
// Normalize the path to ensure it is consistent.
255256
std::string normalized_file_path = NormalizeFileURLOrPath(env, file_path);
256257
if (normalized_file_path.empty()) {
@@ -325,6 +326,10 @@ void CompileCacheHandler::MaybeSaveImpl(CompileCacheEntry* entry,
325326
Debug("keeping the in-memory entry\n");
326327
return;
327328
}
329+
if (read_only_) {
330+
Debug("read-only, not serializing\n");
331+
return;
332+
}
328333
Debug("%s the in-memory entry\n",
329334
entry->cache == nullptr ? "initializing" : "refreshing");
330335

@@ -349,6 +354,9 @@ void CompileCacheHandler::MaybeSave(CompileCacheEntry* entry,
349354

350355
void CompileCacheHandler::MaybeSave(CompileCacheEntry* entry,
351356
std::string_view transpiled) {
357+
if (read_only_) {
358+
return;
359+
}
352360
CHECK(entry->type == CachedCodeType::kStrippedTypeScript);
353361
Debug("[compile cache] saving transpilation cache for %s %s\n",
354362
entry->type_name(),
@@ -381,6 +389,10 @@ void CompileCacheHandler::MaybeSave(CompileCacheEntry* entry,
381389
*/
382390
void CompileCacheHandler::Persist() {
383391
DCHECK(!compile_cache_dir_.empty());
392+
if (read_only_) {
393+
Debug("[compile cache] read-only, skipping persistence\n");
394+
return;
395+
}
384396

385397
// TODO(joyeecheung): do this using a separate event loop to utilize the
386398
// libuv thread pool and do the file system operations concurrently.
@@ -546,10 +558,11 @@ CompileCacheEnableResult CompileCacheHandler::Enable(Environment* env,
546558
cache_tag,
547559
cache_dir_with_tag);
548560

549-
if (!env->permission()->is_granted(
550-
env,
551-
permission::PermissionScope::kFileSystemWrite,
552-
cache_dir_with_tag)) [[unlikely]] {
561+
const bool read_only = HasOption(option, EnableOption::READ_ONLY);
562+
if (!read_only && !env->permission()->is_granted(
563+
env,
564+
permission::PermissionScope::kFileSystemWrite,
565+
cache_dir_with_tag)) [[unlikely]] {
553566
result.message = "Skipping compile cache because write permission for " +
554567
cache_dir_with_tag + " is not granted";
555568
result.status = CompileCacheEnableStatus::FAILED;
@@ -566,25 +579,46 @@ CompileCacheEnableResult CompileCacheHandler::Enable(Environment* env,
566579
return result;
567580
}
568581

569-
fs::FSReqWrapSync req_wrap;
570-
int err = fs::MKDirpSync(
571-
nullptr, &(req_wrap.req), cache_dir_with_tag, 0777, nullptr);
572-
if (is_debug_) {
573-
Debug("[compile cache] creating cache directory %s...%s\n",
582+
if (read_only) {
583+
// A read-only cache is used as found and never created: without the
584+
// directory there is nothing to read.
585+
uv_fs_t stat_req;
586+
int err =
587+
uv_fs_stat(nullptr, &stat_req, cache_dir_with_tag.c_str(), nullptr);
588+
// libuv normalizes st_mode across platforms; the S_ISDIR macro is not on MSVC.
589+
bool is_dir = err == 0 && (stat_req.statbuf.st_mode & S_IFMT) == S_IFDIR;
590+
uv_fs_req_cleanup(&stat_req);
591+
Debug("[compile cache] read-only cache directory %s...%s\n",
574592
cache_dir_with_tag,
575-
err < 0 ? uv_strerror(err) : "success");
576-
}
577-
if (err != 0 && err != UV_EEXIST) {
578-
result.message =
579-
"Cannot create cache directory: " + std::string(uv_strerror(err));
580-
result.status = CompileCacheEnableStatus::FAILED;
581-
return result;
593+
is_dir ? "found" : "not found");
594+
if (!is_dir) {
595+
result.message =
596+
"Cache directory does not exist (read-only): " + cache_dir_with_tag;
597+
result.status = CompileCacheEnableStatus::FAILED;
598+
return result;
599+
}
600+
} else {
601+
fs::FSReqWrapSync req_wrap;
602+
int err = fs::MKDirpSync(
603+
nullptr, &(req_wrap.req), cache_dir_with_tag, 0777, nullptr);
604+
if (is_debug_) {
605+
Debug("[compile cache] creating cache directory %s...%s\n",
606+
cache_dir_with_tag,
607+
err < 0 ? uv_strerror(err) : "success");
608+
}
609+
if (err != 0 && err != UV_EEXIST) {
610+
result.message =
611+
"Cannot create cache directory: " + std::string(uv_strerror(err));
612+
result.status = CompileCacheEnableStatus::FAILED;
613+
return result;
614+
}
582615
}
583616

584617
result.cache_directory = absolute_cache_dir_base;
585618
compile_cache_dir_ = cache_dir_with_tag;
586619
portable_ = option;
587-
if (option == EnableOption::PORTABLE) {
620+
read_only_ = read_only;
621+
if (HasOption(option, EnableOption::PORTABLE)) {
588622
normalized_compile_cache_dir_ =
589623
NormalizeFileURLOrPath(env, compile_cache_dir_);
590624
}

‎src/compile_cache.h‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,22 @@ struct CompileCacheEnableResult {
6060
std::string message; // Set in case of failure.
6161
};
6262

63-
enum class EnableOption : uint8_t { DEFAULT, PORTABLE };
63+
enum class EnableOption : uint8_t {
64+
DEFAULT = 0,
65+
PORTABLE = 1 << 0,
66+
// Only read existing cache entries: nothing is compiled into the in-memory
67+
// store for persisting, nothing is written to disk, and the cache directory
68+
// is not created if it does not exist.
69+
READ_ONLY = 1 << 1,
70+
};
71+
72+
inline constexpr EnableOption operator|(EnableOption a, EnableOption b) {
73+
return static_cast<EnableOption>(static_cast<uint8_t>(a) |
74+
static_cast<uint8_t>(b));
75+
}
76+
inline constexpr bool HasOption(EnableOption value, EnableOption flag) {
77+
return (static_cast<uint8_t>(value) & static_cast<uint8_t>(flag)) != 0;
78+
}
6479

6580
class CompileCacheHandler {
6681
public:
@@ -82,6 +97,7 @@ class CompileCacheHandler {
8297
bool rejected);
8398
void MaybeSave(CompileCacheEntry* entry, std::string_view transpiled);
8499
std::string_view cache_dir() { return compile_cache_dir_; }
100+
bool read_only() const { return read_only_; }
85101

86102
private:
87103
void ReadCacheFile(CompileCacheEntry* entry);
@@ -107,6 +123,7 @@ class CompileCacheHandler {
107123
std::string compile_cache_dir_;
108124
std::string normalized_compile_cache_dir_;
109125
EnableOption portable_ = EnableOption::DEFAULT;
126+
bool read_only_ = false;
110127
std::unordered_map<uint32_t, std::unique_ptr<CompileCacheEntry>>
111128
compiler_cache_store_;
112129
};

‎src/env.cc‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1195,8 +1195,14 @@ void Environment::InitializeCompileCache() {
11951195
DebugCategory::COMPILE_CACHE,
11961196
"[compile cache] using relative path\n");
11971197
}
1198-
EnableCompileCache(dir_from_env,
1199-
portable ? EnableOption::PORTABLE : EnableOption::DEFAULT);
1198+
std::string read_only_env;
1199+
bool read_only = credentials::SafeGetenv(
1200+
"NODE_COMPILE_CACHE_READONLY", &read_only_env, this) &&
1201+
read_only_env == "1";
1202+
EnableOption option = EnableOption::DEFAULT;
1203+
if (portable) option = option | EnableOption::PORTABLE;
1204+
if (read_only) option = option | EnableOption::READ_ONLY;
1205+
EnableCompileCache(dir_from_env, option);
12001206
}
12011207

12021208
CompileCacheEnableResult Environment::EnableCompileCache(

‎src/node_modules.cc‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -508,7 +508,10 @@ void EnableCompileCache(const FunctionCallbackInfo<Value>& args) {
508508

509509
EnableOption option = EnableOption::DEFAULT;
510510
if (args.Length() > 1 && args[1]->IsTrue()) {
511-
option = EnableOption::PORTABLE;
511+
option = option | EnableOption::PORTABLE;
512+
}
513+
if (args.Length() > 2 && args[2]->IsTrue()) {
514+
option = option | EnableOption::READ_ONLY;
512515
}
513516

514517
Utf8Value value(isolate, args[0]);
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
'use strict';
2+
3+
// This tests module.enableCompileCache({ directory, readOnly: true }): existing
4+
// entries are read, nothing is written, and a missing directory is not created.
5+
6+
const common = require('../common');
7+
const { spawnSyncAndAssert } = require('../common/child_process');
8+
const assert = require('assert');
9+
const fs = require('fs');
10+
const path = require('path');
11+
const tmpdir = require('../common/tmpdir');
12+
const fixtures = require('../common/fixtures');
13+
14+
tmpdir.refresh();
15+
const wrapper = fixtures.path('compile-cache-wrapper-options.js');
16+
const target = path.join(tmpdir.path, 'target.js');
17+
fs.writeFileSync(target, 'module.exports = 1;');
18+
const other = path.join(tmpdir.path, 'other.js');
19+
fs.writeFileSync(other, 'module.exports = 2;');
20+
const directory = path.join(tmpdir.path, 'cache');
21+
const list = () => fs.readdirSync(directory, { recursive: true }).sort();
22+
const run = (options, extraEnv, requires, check) => spawnSyncAndAssert(
23+
process.execPath,
24+
[...requires.flatMap((r) => ['-r', r]), target],
25+
{
26+
env: {
27+
...process.env,
28+
NODE_DEBUG_NATIVE: 'COMPILE_CACHE',
29+
NODE_TEST_COMPILE_CACHE_OPTIONS: JSON.stringify(options),
30+
...extraEnv,
31+
},
32+
},
33+
check);
34+
35+
// Read-only against a directory that does not exist: enabling fails and
36+
// nothing is created.
37+
run({ directory, readOnly: true }, {}, [wrapper], {
38+
stderr: /read-only cache directory .*\.\.\.not found/,
39+
});
40+
assert(!fs.existsSync(directory));
41+
42+
// A normal run generates the cache for target.js.
43+
run({ directory }, {}, [wrapper], {
44+
stderr: /writing cache for .*target\.js.*success/,
45+
});
46+
const generated = list();
47+
assert.notStrictEqual(generated.length, 0);
48+
49+
// Read-only against it: target.js's entry is accepted, other.js is compiled
50+
// but not written, and persistence is skipped.
51+
run({ directory, readOnly: true }, {}, [wrapper, other], {
52+
stderr: common.mustCall((output) => {
53+
assert.match(output, /cache for .*target\.js was accepted/);
54+
assert.match(output, /read-only, skipping persistence/);
55+
assert.doesNotMatch(output, /writing cache for/);
56+
return true;
57+
}),
58+
});
59+
assert.deepStrictEqual(list(), generated);
60+
61+
// The environment variable form.
62+
run({ directory }, { NODE_COMPILE_CACHE_READONLY: '1' }, [wrapper, other], {
63+
stderr: /read-only, skipping persistence/,
64+
});
65+
assert.deepStrictEqual(list(), generated);

0 commit comments

Comments
 (0)