Include CTFE benchmark in perf tests

[RalfJung]

Before I mess with CTFE-related things (really I want to mess with miri but one doesn't go without the other), it'd be nice to have benchmarks that ensure performance does not regress.

@oli-obk Do you know about a crate or a bunch of testcases that heavily stress CTFE and would make for a good benchmark?

Also, what would it take to add a new crate to the perf tests?

[oli-obk]

Also, what would it take to add a new crate to the perf tests?

Copy it to https://github.com/rust-lang-nursery/rustc-perf/tree/master/collector/benchmarks

Do you know about a crate or a bunch of testcases that heavily stress CTFE and would make for a good benchmark?

the kekkac part of https://github.com/RustCrypto/sponges is pretty hard on const prop, so that is a good thing to add in general

Other than that I just know of the brotli2 crate. But I can come up with some additional degenerate cases if you want :D

[RalfJung]

the kekkac part of https://github.com/RustCrypto/sponges

There is no non-kekkac part to this?^^

Other than that I just know of the brotli2 crate.

So would it be preferred to just copy these two crates wholesale, or to extract some pieces from them and add those as a single ctfe-benchmarks crate?

I can come up with some additional degenerate cases if you want :D

That would be great. :D

[oli-obk]

So would it be preferred to just copy these two crates wholesale, or to extract some pieces from them and add those as a single ctfe-benchmarks crate?

I think the kekkac folder alone allowed me to reproduce it. Not sure if simplification is of benefit here, but you can try if you want.

Until we get loops and conditions, the following should cover a good swath of features that have the potential for regressing

static SLOW_RELOCATIONS: [&str; 50000] = ["hello", "hello", .... ];
static SLOW_FIELDS: [&i32; 50000] = [&("bar", 42, "foo", 3.14).1, .... ];
static SLOW_CHECKED_INDEX: [u8; 50000] = [b"foomp"[3], ... ];
static UNSIZING: [&[u8]; 50000] = [b"foo", ...];
trait Trait{}
impl Trait for u32 {}
static UNSIZE_TRAIT: [&Trait; 50000] = [&42u32, ...];
static CHAIN: [usize; 50000] = [42i32 as u8 as u64 as i8 as isize as usize, ...];
static OPS: [i32; 50000] = [((((10 >> 1) + 3) * 7) / 2 - 12) << 4, ...];
static FORCE_ALLOC: [i32; 50000] = [*****(&&&&&5), ...];
const fn nop<T>(t: T) -> T { t }
static CONST_FN_BASELINE: [i32; 50000] = [nop(42), nop(43), ...];
const fn inc(i: i32) -> i32 { i + 1 }
static CONST_FN_SIMPLE: [i32; 50000] = [inc(42), inc(43), ...];

[RalfJung]

I like this ... syntax, can we have it? :D

[RalfJung]

I think the kekkac folder alone allowed me to reproduce it. Not sure if simplification is of benefit here, but you can try if you want.

That doesn't answer the question about "copy the crate" vs "make it part of an artificial CTFE-stress crate" though.

[oli-obk]

make it a part if easy enough to do so, copy otherwise.

[RalfJung]

I suspect, however, that these arrays tests are testing more the const generation and maybe LLVM generation, not actual CTFE execution.

[RalfJung]

This has happened with rust-lang/rustc-perf#282, rust-lang/rustc-perf#266 and rust-lang/rustc-perf#265