expected expression, found `<eof>`

[realaravinth]

First of all, thank you for this amazing crate! I've been having a fun playing around with. :)

Last night, CI builds on nightly failed with expected expression, found '<eof>' and I haven't been able to reproduce the issue locally on my machine.

 error: expected expression, found `<eof>`
Error:  --> /home/runner/work/guard/guard/target/debug/build/sailfish-compiler-2f6bfe1de24fe3be/out/templates/index-ec2437792dcba84c:0:1
  |

error: non-statement macro in statement position: include
Error:   --> src/pages/panel/mod.rs:24:10
   |
24 | #[derive(TemplateOnce, Clone)]
   |          ^^^^^^^^^^^^
   |
   = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)

The failure is from a template that wasn't changed in the commits that triggered it.

Here are the logs from the failed CI runs:

• first run
• second run

Let me know if there's something that I can do to help. Thanks!

[ggwpez]

Getting the same error.
Maybe since its using edition = 2018? Somehow it works locally but fails in the CI like you said…
Command is here with rust 1.60.

[ggwpez]

This is still erroring from time to time, another CI example https://github.com/ggwpez/substrate-weight-compare/actions/runs/4237520589/jobs/7363576802

[Kogia-sima]

cannot reproduce an error on Lubuntu 22.04.1 LTS x86_64 (GNU libc, Intel i5-8265U @ 3.900GHz)

$ cargo test --locked --no-run --release --all-targets
    Finished release [optimized] target(s) in 0.15s
  Executable unittests src/main.rs (target/release/deps/swc-6ba302bf6e473a52)
  Executable tests/swc_works.rs (target/release/deps/swc_works-d9fcffd9dedb8761)
  Executable unittests src/lib.rs (target/release/deps/swc_core-91693cf945e06247)
  Executable benches/parse.rs (target/release/deps/parse-552a7381554a2310)
  Executable benches/term.rs (target/release/deps/term-1a358952933f5397)
  Executable unittests src/main.rs (target/release/deps/swc_web-aa912a30e7bf0e9b)
  Executable tests/swc_web_works.rs (target/release/deps/swc_web_works-744358ebcd6edfc4)

[ggwpez]

Yea it is nondeterministic, only happens every now and them. There seems to be no good way to locally re-produce it.
Something about the GitHub CI triggers it more often. Eg another one https://github.com/ggwpez/substrate-weight-compare/actions/runs/4316344945/jobs/7532000895

I assumed it was something with the build cache in combination with Github local actions runner, and that linked MR seems to fix it.

[w-flo]

Probably not too useful info, but I wanted to add that this happens regularly for me when I save one of my project's files in VS Code, which triggers a clippy run through rust-analyzer (error is shown in the "problems" tab). Hitting Ctrl+S again, to trigger another clippy run, fixes the issue. I use a >10 year old SSD so it's not the fastest, but maybe this can rule out the "slow HDD in CI" theory?

I can't reproduce this outside of rust-analyzer though, manually running cargo run / clippy / test etc. never seems to fail. In all of my CI runs (~20 runs on gitea CI on a really slow machine, >10 year old laptop-level hardware that uses external USB HDD for storage), it hasn't failed yet either.

Maybe something both rust-analyzer and github CI have in common?

[ggwpez]

It is indeed something with the filesystem, I fixed it here #111 but that also removes some error logs.
Currently I am using my own fork since I dont have the time to find another solution while keeping the error prints 🤷. But that has the problem of not being on crates,io, so its not a good solution either.

[w-flo]

That's interesting! Trying to reproduce the issue, I noticed that my "cargo clippy through rust-analyzer" issue is not exactly the same, or maybe rust-analyzer somehow prints a partial error message only. I only see this part: "non-statement macro in statement position: include", no <eof> error message. But it's probably related or the same issue.

And while trying to figure out what exactly that error is trying to tell me, I found this: rust-lang/rust#41497

And it turns out the "non-statement macro in statement position: include" error along with the <eof> error can be reproduced like this:
main.rs:

fn main() {
    include!("b.rs");
}

and b.rs should be an empty 0-byte file (touch src/b.rs)

So there are probably two options:

• the out/templates/... file is actually (still/again?) empty for some reason when the code generated by the sailfish proc macro is executed
• the file is not empty, but include!() thinks it is empty

[ggwpez]

• the out/templates/... file is actually (still/again?) empty for some reason when the code generated by the sailfish proc macro is executed

Yea I think there is just some race condition with the file system, since the proc macro generates a file and then reads it later.
Maybe it is enough to flush the IO or something, did not try.

[w-flo]

I think drop(f) as in the current code should be enough to flush the file at least for the current process (it might be a problem if rust at some point decides to run proc macros in separate processes? Not sure what guarantees OSes / filesystems usually give for file system consistency for read-after-write from different processes, and I believe there is no guarantee that sailfish proc macro execution will happen in the same process that will execute the include!() macro).

Anyway, I might have figured out the problem I was encountering when using rust-analyzer. It turns out that rust-analyzer calls cargo clippy with the "--all-targets" flag, which results in at least two check processes (apparently one for my library, and one for my test suite? Not sure what exactly is going on there) running at the same time. I added a simple eprintln!() to the sailfish proc macro and sure enough, there's a race condition where apparently two processes both write the same sailfish compilation result to the same file and then try to include!() the file shortly afterwards, which fails every now and then:

compiled to file "[snip]/target/debug/build/sailfish-compiler-b6ddc1e6967b5596/out/templates/login-d9afabe3f026b7df"
compiled to file "[snip]/target/debug/build/sailfish-compiler-b6ddc1e6967b5596/out/templates/login-d9afabe3f026b7df"
[... two lines for every template file I use]

When running just "cargo clippy" without the --all-targets flag, there is only one line of output for each template, so no race condition.

I'm not sure about the best approach to fix it. Add some more relevant data from the environment to the filename hash, like the "build type" if that is available? Or maybe add a random number instead of the hash, and then somehow identify and remove stale files on next run?

[w-flo]

To address the "needs reproduction" tag, this works for me:

cargo new repro
cd repro/
cargo add sailfish
mkdir templates
for i in {1..50}; do echo "<% for x in 0..100 { %> <%= x %> <% } %>" >> templates/repro.stpl; done
echo "#[derive(sailfish::TemplateOnce)]#[template(path = \"repro.stpl\")] struct Repro; fn main() {}" > src/main.rs
while true; do touch src/main.rs; cargo clippy --all-targets || break; done

The loop will eventually hit an error on my setup, but since this is a race condition, maybe it won't work for everyone. In my actual project, it appears to be more common (maybe simply because it uses more than 1 template).

When using the linked draft PR, the bug appears to be fixed for me. And these no-change clippy runs take ~0.13 seconds instead of ~0.17 seconds, so that's cool.