lsif-go OOMing in CI

[chrismwendt]

• First reported in Slack
• Probably caused by the find-implementations PR loading more packages Add support for find implementations lsif-go#199
• Implementations was included in lsif-go v1.7.0 and v1.7.1 https://github.com/sourcegraph/lsif-go/releases

Immediate remediation:

• Downgrade to lsif-go v1.6.7 in CI https://github.com/sourcegraph/sourcegraph/pull/27213

Next steps:

• Try loading less info for packages ❌ Didn't work. I tried a few different subsets of loadMode https://github.com/sourcegraph/lsif-go/blob/e6375badf357cf7dec69be82598fcd571de9c271/internal/indexer/indexer.go#L229 but they all caused segfaults or panics.
• See if we can get more memory in CI. Would probably have to set up a dedicated Azure runner or something. No action taken.
• Add flag to optionally load all deps. If you don't load deps, you don't get cross-repo implementations data. Add --skip-deps flag to reduce memory usage lsif-go#212
• Create issue for loading one dep at a time and extract implementation data from it Load one dependency at a time for implementations lsif-go#213

Notes

Wow, mem usage jumped from 200MB to 1.4GB:

• v1.6.7: Alloc = 209 MiB, Sys = 315 MiB
• v1.7.1: Alloc = 1423 MiB, Sys = 2246 MiB

The measurement was taken immediately after loadPackages in gorilla/mux.

For sourcegraph/sourcegraph: 6.6GB of Alloc, 12GB Sys 💥 That exceeds GH Actions runners 7GB memory. Prior to implementations, it used 2.2GB Alloc, 4GB Sys.

With loadMode2 below, it's 5.8GB of Alloc.

var loadMode = packages.NeedDeps | packages.NeedFiles | packages.NeedImports | packages.NeedSyntax | packages.NeedTypes | packages.NeedTypesInfo | packages.NeedName

var loadMode2 = packages.NeedDeps | packages.NeedTypes | packages.NeedTypesInfo

Removing NeedDeps drops perf (I don't know why), and lsif-go really needs types, so this seems to be the minimal subset.

Loading deps one at a time is suuuper slow, ~1s each and there are 1345 deps of sourcegraph/sourcegraph, so it would take ~22 minutes total. Loading all at once takes 15s.

[chrismwendt]

Resolved by skipping loading deps by default in lsif-go sourcegraph/lsif-go#212