Skip to content

x/tools/gopls: high memory consumption on 19MLoC input #73709

Open
@bytemain

Description

@bytemain

gopls version

Build info
----------
golang.org/x/tools/gopls v0.18.1
    golang.org/x/tools/gopls@v0.18.1 h1:2xJBNzdImS5u/kV/ZzqDLSvlBSeZX+pWY9uKVP7Pask=
    github.com/BurntSushi/toml@v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=
    github.com/google/go-cmp@v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
    golang.org/x/exp/typeparams@v0.0.0-20241210194714-1829a127f884 h1:1xaZTydL5Gsg78QharTwKfA9FY9CZ1VQj6D/AZEvHR0=
    golang.org/x/mod@v0.23.0 h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM=
    golang.org/x/sync@v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w=
    golang.org/x/telemetry@v0.0.0-20241220003058-cc96b6e0d3d9 h1:L2k9GUV2TpQKVRGMjN94qfUMgUwOFimSQ6gipyJIjKw=
    golang.org/x/text@v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=
    golang.org/x/tools@v0.30.1-0.20250221230316-5055f70f240c h1:Ja/5gV5a9Vvho3p2NC/T2TtxhHjrWS/2DvCKMvA0a+Y=
    golang.org/x/vuln@v1.1.3 h1:NPGnvPOTgnjBc9HTaUx+nj+EaUYxl5SJOWqaDYGaFYw=
    honnef.co/go/tools@v0.5.1 h1:4bH5o3b5ZULQ4UrBmP+63W9r7qIkqJClEA9ko5YKx+I=
    mvdan.cc/gofumpt@v0.7.0 h1:bg91ttqXmi9y2xawvkuMXyvAA/1ZGJqYAEGjXuP0JXU=
    mvdan.cc/xurls/v2@v2.5.0 h1:lyBNOm8Wo71UknhUs4QTFUNNMyxy2JEIaKKo0RWOh+8=
go: go1.23.9

go env

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/bytedance/Library/Caches/go-build'
GOENV='/Users/bytedance/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/bytedance/go/pkg/mod'
GOOS='darwin'
GOPATH='/Users/bytedance/go'
GOPROXY='https://goproxy.cn|direct'
GOROOT='/Users/bytedance/sdk/go1.22.12'
GOSUMDB='sum.golang.google.cn'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/bytedance/sdk/go1.22.12/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.12'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='[redacted]/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/fz/2x5883ld6c98wqhkgkzyxp_c0000gn/T/go-build8745845=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

Open project in vscode, when first time open, it will consume nearly 40gb memory.

it can be reproduced if clean gopls cache and open the program again:

rm -rf ~/Library/Caches/gopls

VSCode says it Analyzing Dependencies.

Image

What did you see happen?

Image

and the UI of VSCode is very laggy

What did you expect to see?

Normal memory consumption.

Editor and settings

{
    "database-client.autoSync": true,
    "netbeans.javaSupport.enabled": false,
    "vsicons.dontShowNewVersionMessage": true,
    "makefile.configureOnOpen": true,
    "go.languageServerFlags": [
        "-debug=localhost:8080",
        "-logfile",
        "-rpc.trace"
    ]
}

Logs

gopls stats
Initializing workspace...     done (15.325508083s)
Gathering bug reports...      done (609.709ms)
Querying memstats...          done (261.94525ms)
Querying workspace stats...   done (10.897667ms)
Collecting directory info...  done (15.071791ms)
{
  "BugReports": [],
  "CacheDir": "/Users/bytedance/Library/Caches/gopls/c65eb5cd",
  "DirStats": {
    "Files": 369,
    "TestdataFiles": 0,
    "GoFiles": 319,
    "ModFiles": 2,
    "Dirs": 152
  },
  "GOARCH": "arm64",
  "GOOS": "darwin",
  "GOPACKAGESDRIVER": "",
  "GOPLSCACHE": "",
  "GoVersion": "go1.23.9",
  "GoplsVersion": "v0.18.1",
  "InitialWorkspaceLoadDuration": "15.325508083s",
  "MemStats": {
    "HeapAlloc": 1763848432,
    "HeapInUse": 1901535232,
    "TotalAlloc": 35196170592
  },
  "WorkspaceStats": {
    "Files": {
      "Total": 14909,
      "Largest": 12682118,
      "Errs": 0
    },
    "Views": [
      {
        "GoCommandVersion": "go1.22.12",
        "AllPackages": {
          "Packages": 3290,
          "LargestPackage": 293,
          "CompiledGoFiles": 14933,
          "Modules": 574
        },
        "WorkspacePackages": {
          "Packages": 120,
          "LargestPackage": 29,
          "CompiledGoFiles": 339,
          "Modules": 1
        },
        "Diagnostics": 42
      }
    ]
  }
}

Allocated bytes 29,696,939,920
Total allocated bytes 83,021,201,064
System bytes 31,415,423,896
Heap system bytes 30,857,355,264
Malloc calls 639,069,318
Frees 331,351,033
Idle heap bytes 890,183,680
In use bytes 29,967,171,584
Released to system bytes 15,228,928
Heap object count 307,718,285
Stack in use bytes 3,571,712
Stack from system bytes 3,571,712
Bucket hash bytes 11,253,092
GC metadata bytes 225,733,064
Off heap bytes 30,899,644

and cannot see the gopls.1234-5GiB-withnames.zip file in $TMPDIR

Image

As it is the company's code, I handled some sensitive information in the log.

gopls.log

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugReportIssues describing a possible bug in the Go implementation.ToolsThis label describes issues relating to any tools in the x/tools repository.goplsIssues related to the Go language server, gopls.gopls/performanceIssues related to gopls performance (CPU, memory, etc).

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions