Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bootstrap: put is_building_snapshot state in IsolateData and introduce ERR_NOT_SUPPORTED_IN_SNAPSHOT #47887

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions doc/api/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -2414,6 +2414,13 @@ error indicates that the idle loop has failed to stop.
An attempt was made to use operations that can only be used when building
V8 startup snapshot even though Node.js isn't building one.

<a id="ERR_NOT_SUPPORTED_IN_SNAPSHOT"></a>

### `ERR_NOT_SUPPORTED_IN_SNAPSHOT`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about using a more generic ERR_NOT_SUPPORTED error? The concrete description could be part of the message. We have many very specific error codes that likely do not help users all that much.

Copy link
Member Author

@joyeecheung joyeecheung May 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think a generic ERR_NOT_SUPPORTED would be quite useful in this case. What would one do after checking error.code === ERR_NOT_SUPPORTED? I can imagine the embedder putting a TryCatch around e.g. LoadEnvironment and check for the code being ERR_NOT_SUPPORTED_IN_SNAPSHOT, if that's true, simply give up creating a snapshot (as some kind of optional optimization) and always run the code from scratch, otherwise simply treat it as a proper error (e.g. the code being run is actually buggy) that it cannot handle and fail. ERR_NOT_SUPPORTED would not be enough in this case, and then they'd end up..parsing the error message to see if it's a snapshot failure.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping @BridgeAR does that answer the question?


An attempt was made to perform operations that are not supported when
building a startup snapshot.

<a id="ERR_NO_CRYPTO"></a>

### `ERR_NO_CRYPTO`
Expand Down
2 changes: 2 additions & 0 deletions lib/internal/errors.js
Original file line number Diff line number Diff line change
Expand Up @@ -1464,6 +1464,8 @@ E('ERR_NETWORK_IMPORT_DISALLOWED',
"import of '%s' by %s is not supported: %s", Error);
E('ERR_NOT_BUILDING_SNAPSHOT',
'Operation cannot be invoked when not building startup snapshot', Error);
E('ERR_NOT_SUPPORTED_IN_SNAPSHOT',
'%s is not supported in startup snapshot', Error);
E('ERR_NO_CRYPTO',
'Node.js is not compiled with OpenSSL crypto support', Error);
E('ERR_NO_ICU',
Expand Down
12 changes: 10 additions & 2 deletions lib/internal/v8/startup_snapshot.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ const {
const {
codes: {
ERR_NOT_BUILDING_SNAPSHOT,
ERR_NOT_SUPPORTED_IN_SNAPSHOT,
ERR_DUPLICATE_STARTUP_SNAPSHOT_MAIN_FUNCTION,
},
} = require('internal/errors');
Expand All @@ -14,11 +15,11 @@ const {
setSerializeCallback,
setDeserializeCallback,
setDeserializeMainFunction: _setDeserializeMainFunction,
isBuildingSnapshotBuffer,
} = internalBinding('mksnapshot');

function isBuildingSnapshot() {
// For now this is the only way to build a snapshot.
return require('internal/options').getOptionValue('--build-snapshot');
return isBuildingSnapshotBuffer[0];
}

function throwIfNotBuildingSnapshot() {
Expand All @@ -27,6 +28,12 @@ function throwIfNotBuildingSnapshot() {
}
}

function throwIfBuildingSnapshot(reason) {
if (isBuildingSnapshot()) {
throw new ERR_NOT_SUPPORTED_IN_SNAPSHOT(reason);
}
}

const deserializeCallbacks = [];
let deserializeCallbackIsSet = false;
function runDeserializeCallbacks() {
Expand Down Expand Up @@ -102,6 +109,7 @@ function setDeserializeMainFunction(callback, data) {
module.exports = {
initializeCallbacks,
runDeserializeCallbacks,
throwIfBuildingSnapshot,
// Exposed to require('v8').startupSnapshot
namespace: {
addDeserializeCallback,
Expand Down
5 changes: 4 additions & 1 deletion lib/internal/worker.js
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,9 @@ const { deserializeError } = require('internal/error_serdes');
const { fileURLToPath, isURL, pathToFileURL } = require('internal/url');
const { kEmptyObject } = require('internal/util');
const { validateArray, validateString } = require('internal/validators');

const {
throwIfBuildingSnapshot,
} = require('internal/v8/startup_snapshot');
const {
ownsProcessState,
isMainThread,
Expand Down Expand Up @@ -129,6 +131,7 @@ function assignEnvironmentData(data) {

class Worker extends EventEmitter {
constructor(filename, options = kEmptyObject) {
throwIfBuildingSnapshot('Creating workers');
super();
const isInternal = arguments[2] === kIsInternal;
debug(
Expand Down
4 changes: 2 additions & 2 deletions src/api/embed_helpers.cc
Original file line number Diff line number Diff line change
Expand Up @@ -142,8 +142,8 @@ CommonEnvironmentSetup::CommonEnvironmentSetup(

impl_->isolate_data.reset(CreateIsolateData(
isolate, loop, platform, impl_->allocator.get(), snapshot_data));
impl_->isolate_data->options()->build_snapshot =
impl_->snapshot_creator.has_value();
impl_->isolate_data->set_is_building_snapshot(
impl_->snapshot_creator.has_value());

if (snapshot_data) {
impl_->env.reset(make_env(this));
Expand Down
1 change: 1 addition & 0 deletions src/base_object_types.h
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ namespace node {
#define SERIALIZABLE_BINDING_TYPES(V) \
V(encoding_binding_data, encoding_binding::BindingData) \
V(fs_binding_data, fs::BindingData) \
V(mksnapshot_binding_data, mksnapshot::BindingData) \
V(v8_binding_data, v8_utils::BindingData) \
V(blob_binding_data, BlobBindingData) \
V(process_binding_data, process::BindingData) \
Expand Down
4 changes: 4 additions & 0 deletions src/env.h
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,9 @@ class NODE_EXTERN_PRIVATE IsolateData : public MemoryRetainer {
void MemoryInfo(MemoryTracker* tracker) const override;
IsolateDataSerializeInfo Serialize(v8::SnapshotCreator* creator);

bool is_building_snapshot() const { return is_building_snapshot_; }
void set_is_building_snapshot(bool value) { is_building_snapshot_ = value; }

inline uv_loop_t* event_loop() const;
inline MultiIsolatePlatform* platform() const;
inline const SnapshotData* snapshot_data() const;
Expand Down Expand Up @@ -219,6 +222,7 @@ class NODE_EXTERN_PRIVATE IsolateData : public MemoryRetainer {
const SnapshotData* snapshot_data_;
std::shared_ptr<PerIsolateOptions> options_;
worker::Worker* worker_context_ = nullptr;
bool is_building_snapshot_ = false;
};

struct ContextInfo {
Expand Down
4 changes: 2 additions & 2 deletions src/node.cc
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,7 @@ MaybeLocal<Value> StartExecution(Environment* env, StartExecutionCallback cb) {
auto reset_entry_point =
OnScopeLeave([&]() { env->set_embedder_entry_point({}); });

const char* entry = env->isolate_data()->options()->build_snapshot
const char* entry = env->isolate_data()->is_building_snapshot()
? "internal/main/mksnapshot"
: "internal/main/embedding";

Expand Down Expand Up @@ -311,7 +311,7 @@ MaybeLocal<Value> StartExecution(Environment* env, StartExecutionCallback cb) {
return StartExecution(env, "internal/main/inspect");
}

if (env->isolate_data()->options()->build_snapshot) {
if (env->isolate_data()->is_building_snapshot()) {
return StartExecution(env, "internal/main/mksnapshot");
}

Expand Down
1 change: 1 addition & 0 deletions src/node_binding.h
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ static_assert(static_cast<int>(NM_F_LINKED) ==
V(contextify) \
V(encoding_binding) \
V(fs) \
V(mksnapshot) \
V(timers) \
V(process_methods) \
V(performance) \
Expand Down
2 changes: 2 additions & 0 deletions src/node_main_instance.cc
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ NodeMainInstance::NodeMainInstance(const SnapshotData* snapshot_data,
platform,
array_buffer_allocator_.get(),
snapshot_data->AsEmbedderWrapper().get()));
isolate_data_->set_is_building_snapshot(
per_process::cli_options->per_isolate->build_snapshot);

isolate_data_->max_young_gen_size =
isolate_params_->constraints.max_young_generation_size_in_bytes();
Expand Down
101 changes: 89 additions & 12 deletions src/node_snapshotable.cc
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
#include <iostream>
#include <sstream>
#include <vector>
#include "aliased_buffer-inl.h"
#include "base_object-inl.h"
#include "debug_utils-inl.h"
#include "encoding_binding.h"
Expand Down Expand Up @@ -33,6 +34,7 @@ namespace node {
using v8::Context;
using v8::Function;
using v8::FunctionCallbackInfo;
using v8::FunctionTemplate;
using v8::HandleScope;
using v8::Isolate;
using v8::Local;
Expand Down Expand Up @@ -1498,8 +1500,6 @@ void SerializeSnapshotableObjects(Realm* realm,
});
}

namespace mksnapshot {

// NB: This is also used by the regular embedding codepath.
void GetEmbedderEntryFunction(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
Expand Down Expand Up @@ -1572,16 +1572,89 @@ void SetDeserializeMainFunction(const FunctionCallbackInfo<Value>& args) {
env->set_snapshot_deserialize_main(args[0].As<Function>());
}

void Initialize(Local<Object> target,
Local<Value> unused,
Local<Context> context,
void* priv) {
namespace mksnapshot {

BindingData::BindingData(Realm* realm,
v8::Local<v8::Object> object,
InternalFieldInfo* info)
: SnapshotableObject(realm, object, type_int),
is_building_snapshot_buffer_(
realm->isolate(),
1,
MAYBE_FIELD_PTR(info, is_building_snapshot_buffer)) {
if (info == nullptr) {
object
->Set(
realm->context(),
FIXED_ONE_BYTE_STRING(realm->isolate(), "isBuildingSnapshotBuffer"),
is_building_snapshot_buffer_.GetJSArray())
.Check();
} else {
is_building_snapshot_buffer_.Deserialize(realm->context());
}
// Reset the status according to the current state of the realm.
bool is_building_snapshot = realm->isolate_data()->is_building_snapshot();
DCHECK_IMPLIES(is_building_snapshot,
realm->isolate_data()->snapshot_data() == nullptr);
is_building_snapshot_buffer_[0] = is_building_snapshot ? 1 : 0;
is_building_snapshot_buffer_.MakeWeak();
}

bool BindingData::PrepareForSerialization(Local<Context> context,
v8::SnapshotCreator* creator) {
DCHECK_NULL(internal_field_info_);
internal_field_info_ = InternalFieldInfoBase::New<InternalFieldInfo>(type());
internal_field_info_->is_building_snapshot_buffer =
is_building_snapshot_buffer_.Serialize(context, creator);
// Return true because we need to maintain the reference to the binding from
// JS land.
return true;
}

InternalFieldInfoBase* BindingData::Serialize(int index) {
DCHECK_EQ(index, BaseObject::kEmbedderType);
InternalFieldInfo* info = internal_field_info_;
internal_field_info_ = nullptr;
return info;
}

void BindingData::Deserialize(Local<Context> context,
Local<Object> holder,
int index,
InternalFieldInfoBase* info) {
DCHECK_EQ(index, BaseObject::kEmbedderType);
v8::HandleScope scope(context->GetIsolate());
Realm* realm = Realm::GetCurrent(context);
// Recreate the buffer in the constructor.
InternalFieldInfo* casted_info = static_cast<InternalFieldInfo*>(info);
BindingData* binding =
realm->AddBindingData<BindingData>(context, holder, casted_info);
CHECK_NOT_NULL(binding);
}

void BindingData::MemoryInfo(MemoryTracker* tracker) const {
tracker->TrackField("is_building_snapshot_buffer",
is_building_snapshot_buffer_);
}

void CreatePerContextProperties(Local<Object> target,
Local<Value> unused,
Local<Context> context,
void* priv) {
Realm* realm = Realm::GetCurrent(context);
realm->AddBindingData<BindingData>(context, target);
}

void CreatePerIsolateProperties(IsolateData* isolate_data,
Local<FunctionTemplate> ctor) {
Isolate* isolate = isolate_data->isolate();
Local<ObjectTemplate> target = ctor->PrototypeTemplate();
SetMethod(
context, target, "getEmbedderEntryFunction", GetEmbedderEntryFunction);
SetMethod(context, target, "compileSerializeMain", CompileSerializeMain);
SetMethod(context, target, "setSerializeCallback", SetSerializeCallback);
SetMethod(context, target, "setDeserializeCallback", SetDeserializeCallback);
SetMethod(context,
isolate, target, "getEmbedderEntryFunction", GetEmbedderEntryFunction);
SetMethod(isolate, target, "compileSerializeMain", CompileSerializeMain);
SetMethod(isolate, target, "setSerializeCallback", SetSerializeCallback);
SetMethod(isolate, target, "setDeserializeCallback", SetDeserializeCallback);
SetMethod(isolate,
target,
"setDeserializeMainFunction",
SetDeserializeMainFunction);
Expand All @@ -1595,8 +1668,12 @@ void RegisterExternalReferences(ExternalReferenceRegistry* registry) {
registry->Register(SetDeserializeMainFunction);
}
} // namespace mksnapshot

} // namespace node

NODE_BINDING_CONTEXT_AWARE_INTERNAL(mksnapshot, node::mksnapshot::Initialize)
NODE_BINDING_CONTEXT_AWARE_INTERNAL(
mksnapshot, node::mksnapshot::CreatePerContextProperties)
NODE_BINDING_PER_ISOLATE_INIT(mksnapshot,
node::mksnapshot::CreatePerIsolateProperties)
NODE_BINDING_EXTERNAL_REFERENCE(mksnapshot,
node::mksnapshot::RegisterExternalReferences)
26 changes: 26 additions & 0 deletions src/node_snapshotable.h
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

#if defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS

#include "aliased_buffer.h"
#include "base_object.h"
#include "util.h"

Expand Down Expand Up @@ -121,6 +122,31 @@ void DeserializeNodeInternalFields(v8::Local<v8::Object> holder,
void SerializeSnapshotableObjects(Realm* realm,
v8::SnapshotCreator* creator,
RealmSerializeInfo* info);

namespace mksnapshot {
class BindingData : public SnapshotableObject {
public:
struct InternalFieldInfo : public node::InternalFieldInfoBase {
AliasedBufferIndex is_building_snapshot_buffer;
};

BindingData(Realm* realm,
v8::Local<v8::Object> obj,
InternalFieldInfo* info = nullptr);
SET_BINDING_ID(mksnapshot_binding_data)
SERIALIZABLE_OBJECT_METHODS()

void MemoryInfo(MemoryTracker* tracker) const override;
SET_SELF_SIZE(BindingData)
SET_MEMORY_INFO_NAME(BindingData)

private:
AliasedUint8Array is_building_snapshot_buffer_;
InternalFieldInfo* internal_field_info_ = nullptr;
};

} // namespace mksnapshot

} // namespace node

#endif // defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
Expand Down
2 changes: 2 additions & 0 deletions src/node_worker.cc
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ class WorkerThreadData {
allocator.get(),
w->snapshot_data()->AsEmbedderWrapper().get()));
CHECK(isolate_data_);
CHECK(!isolate_data_->is_building_snapshot());
if (w_->per_isolate_opts_)
isolate_data_->set_options(std::move(w_->per_isolate_opts_));
isolate_data_->set_worker_context(w_);
Expand Down Expand Up @@ -481,6 +482,7 @@ void Worker::New(const FunctionCallbackInfo<Value>& args) {
THROW_ERR_MISSING_PLATFORM_FOR_WORKER(env);
return;
}
CHECK(!env->isolate_data()->is_building_snapshot());
anonrig marked this conversation as resolved.
Show resolved Hide resolved

std::string url;
std::string name;
Expand Down
5 changes: 5 additions & 0 deletions test/fixtures/snapshot/worker.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
'use strict';

const { Worker } = require('worker_threads');

new Worker('1', { eval: true });
34 changes: 34 additions & 0 deletions test/parallel/test-snapshot-worker.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
'use strict';

// This tests snapshot JS API using the example in the docs.

require('../common');
const assert = require('assert');
const { spawnSync } = require('child_process');
const tmpdir = require('../common/tmpdir');
const fixtures = require('../common/fixtures');
const path = require('path');
const fs = require('fs');

tmpdir.refresh();
const blobPath = path.join(tmpdir.path, 'snapshot.blob');
const entry = fixtures.path('snapshot', 'worker.js');
{
const child = spawnSync(process.execPath, [
'--snapshot-blob',
blobPath,
'--build-snapshot',
entry,
], {
cwd: tmpdir.path
});
const stderr = child.stderr.toString();
assert.match(
stderr,
/Error: Creating workers is not supported in startup snapshot/);
assert.match(
stderr,
/ERR_NOT_SUPPORTED_IN_SNAPSHOT/);
assert.strictEqual(child.status, 1);
assert(!fs.existsSync(blobPath));
}