Resource Embedding

[spotandjake]

It would be nice if we could import files in grain as just raw bytes at compile time.

Zig seems todo this with comptime and it is just a function @embedFile https://ziglang.org/documentation/master/#embedFile

Rust seems to use a macro for this: https://doc.rust-lang.org/std/macro.include_bytes.html

One idea would be to use syntax like from "filepath" embed x, this complements our include syntax really nicely. alternatively maybe we just do embed "file" as x though this raises questions in relation to the as keyword given its usually used for aliasing.

This feature is useful in a lot of places especially game development where you would be including resources such as images or music.

for now this feature can just be implemented as syntax sugar that generates a byte string bassically:

from "filepath" embed * as x

would get converted to:

let x = b"<fileContents>"

probably somewhere in our middle end phase before typechecking.
in the future once we have datasections it might look into implementing this directly using those.

[tjpalmer]

Syntax like from "filepath" embed x makes it harder to use as an arbitrary expression, unlike the other macros you mentioned. But maybe that's ok since you could just say x later. And I recall some js bundlers doing similar things with imports of external resources.

But there's also the issue that I read from "module" embed x as picking something named x from inside the module, rather than being the whole thing. So, the alias discussion above makes embed "filepath" as x more appropriate to me. So here are a variety of similar options as statements:

• from "filepath" embed x <- Looks like it's getting x from inside the file, which it isn't.
• embed "filepath" as x <- Communicates alias well, but does this add new contextless keyword?
• from "filepath" embed as x <- Communicates alias and avoids contextless keyword, but wordy and awkward?

Or could just go for this as an expression:

• embed "filepath" <- Again, a contextless keyword, and does order of operations matter?
  • Example statement to match above examples: let x = embed "filepath"

NOTE: I heavily edited this comment after reading better. Also to make more readable lists of options to consider.

[tjpalmer]

An example of importing an asset in js with vite:

import imgUrl from './img.png'

Note that is might or might not embed the asset in the js bundle, depending on the size and whatnot. Although I agree that a basic "please make sure to embed" makes extra sense in wasm or native binaries, since despite wasi, there are fewer expectations around runtime access to external files than what you get in js.

[tjpalmer]

And raw string import is also a thing in vite:

import shaderString from './shader.glsl?raw'

Again, maybe don't need the same level of fancy if the primary use case is just embedding a byte array.