Skip to content

Per-version hashing information #1126

Closed

Description

Since the merge of rust-lang/rust#89836, we are seeing high variance on the regression-31157-opt benchmark:

image

This is likely due to hashing being effectively randomized, affecting some hash table layout (I've not narrowed down where/what exactly).

I suspect this may be creating noise on other benchmarks, as well. Our significance algorithm should learn to cope with this in another ~20 commits (roughly 2-3 days) as our backwards window is for 50 commits, but it likely makes sense to try to avoid this problem at the root, and in general it'll make regression triage harder by increasing the baseline changes commit to commit even with small code deltas.

I am not sure yet whether a revert of rust-lang/rust#89836 makes sense, or whether we should use RUSTC_FORCE_INCR_COMP_ARTIFACT_HEADER on rustc-perf to reduce noise levels... in the meantime I'll set the env variable for perf and that'll hopefully eliminate noise in the meantime.

cc @jyn514

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions