Allow applying static code-generation to the published package (i.e. `package.rs`)

[epage]

Problem

When code-generation happens within build.rs or proc macros,

• Arbitrary code is being run on the developer's machine
• The generated code is harder to audit
• Code-generator needs to be built by all dependents when the result is the same

Proposed Solution

A package.rs that runs like build.rs during local development but the output gets captured on cargo publish and it, along with its dependencies, are dropped

If this were combined with a .crate differ on crates.io, it would be easy for dependents (or even package maintainers) to audit the results.

Notes

Past discussions

• Internals: "package.rs script to reduce build dependencies of published crates"
• zulip: "publish.rs"

Alternatives

• code-gen via snapshot testing which has more process overhead which is unappealing for high-churn code-gen

Cases not covered:

• Local development of a package that uses build.rs or a proc macro still has to deal with arbitrary code execution and not easily seeing what gets generated

Complications

• Knowing which dependencies can be stripped
• Inability for dependents to control versions of the code-generator to get the result of bug fixes, requiring a new release
• proc-macros that have tight coupling between the macro and the package that re-exports it
• Capturing proc-macro expansion
• Not subject to feature flags

[epage]

From zulip:

If we sandboxed build.rs with wasm, and we could tell whether it did anything non-deterministic (e.g. read the target, read the time, touched the network, touched files outside the crate directory), then we could pre-run and cache the results of deterministic build scripts and ship that in the published crate.

and

If people need to mix deterministic and non-deterministic tasks in their build.rs, we could use metadeps to isolate those so we'll know which are deterministic and drop their parts of the dependency tree from even running.

[weihanglo]

This needs an RFC anyway.

A publish.rs that runs like build.rs during local development

Alternatively, don't run arbitrary Rust code. Instead provides a set of predefined declarative instructions, so we never require a sandboxed environment. I don't know how it would look like though, given people do weird things during code generations.

[epage]

Yes, this would require an RFC. I just wanted a place to capture this thought until one is written.

[matthieu-m]

Alternatively, don't run arbitrary Rust code. Instead provides a set of predefined declarative instructions, so we never require a sandboxed environment. I don't know how it would look like though, given people do weird things during code generations.

-sys crates would like a word... the build.rs of -sys crates can get very complicated, there's just so much logic to capture depending on the environment.

I can sandboxing the execution (whether with WASM or not), but creating a declarative language to build the native dependencies is a pipe dream.