Use Data.HashMap.lookupKey and Data.HashSet.lookupElement

[sjakobi]

No description provided.

[sjakobi]

I assume the two failing Windows jobs are due to flaky tests?!

[fendor]

@sjakobi Yes, our tests are unfortunately flaky.

[sjakobi]

The bench_example (9.10, ubuntu-latest, 3.14, lsp-types) job is all failures for some reason – the success column is all False:

https://github.com/haskell/haskell-language-server/actions/runs/22140687209/job/64005820648?pr=4802#step:7:1

[sjakobi]

After struggling to get robust results from local runs of the bench/ benchmarks, I've asked Codex to summarize the allocation numbers from the bench_examples CI jobs:

• cabal / ghc 9.10: overall allocation improved.
• Mean delta: -7.07% (median -0.21%), from bench-results/pr-4802/run-22140687209/combined/
  results.csv.
• Biggest wins: code actions -75.25% (17922MB -> 4435MB), completions -33.62%, eval execute
  multi-line code lens -11.14%.
• Biggest regressions: hover after edit +14.81%, completions after edit +13.66%.
• cabal / ghc 9.8: roughly flat to slightly better.
• Mean delta: -0.87% (median +0.91%).
• Biggest win: completions -23.03%.
• Biggest regressions: documentSymbols after edit +5.95%, code actions +5.67%.
• lsp-types / ghc 9.8: overall worse allocation.
• Mean delta: +7.25% (median +0.90%).
• Biggest regressions: hover after edit +60.45% (56459MB -> 90586MB), code actions +54.25%
  (14390MB -> 22196MB), edit +16.92%.
• Biggest win: completions -28.58%.
• lsp-types / ghc 9.10: all benchmark rows are success=False, so allocation deltas there are
  not reliable for performance conclusions.

Is this a somewhat reliable way to assess HLS performance, @fendor? Otherwise, can you suggest a better way?

[fendor]

@sjakobi Thanks, this looks promising! I'd like to look into this huge regression, this seems 15%, seems like it shouldn't be related.

I imagine the reason is that lsp-types-2.1.1.0 can't be built with GHC 9.10? Requires a --allow-newer=template-haskell on my machine. We probably should bump the versions for the benchmark examples... And spend more time on making sure these benchmarks are run regularly.