Go: download step not downloading dependencies?

[fsouza]

👋 While trying pre-commit-ci in a Go project, golangci-lint fails with the following output:

golangci-lint............................................................Failed
- hook id: golangci-lint
- exit code: 3

ERRO Running error: context loading failed: failed to load packages: failed to load with go/packages: err: exit status 1: stderr: go: github.com/gempir/go-twitch-irc/v2@v2.5.0: Get "https://proxy.golang.org/github.com/gempir/go-twitch-irc/v2/@v/v2.5.0.mod": dial tcp: lookup proxy.golang.org on [::1]:53: read udp [::1]:47976->[::1]:53: read: connection refused

Fix End of Files.........................................................Passed
Trim Trailing Whitespace.................................................Passed

This means the download step didn't download the dependencies listed in go.mod. Could it run go mod download (persisting GOMODCACHE between steps) in the presence of a file called go.mod?

Link to run: https://results.pre-commit.ci/run/github/282973629/1607176801.q8Ki4H_SR3ucby0Q2rvnyQ

[asottile]

so by design, pre-commit doesn't install anything from the repository under test -- it looks like parts of gloang-ci-lint depend on the actually repository being "installed" -- you can reproduce the same thing:

set -euxo pipefail
rm -rf twitchbot pc
git clone git@github.com:ChicoCodes/twitchbot
mkdir pc
# if you're using podman, both of these need `--userns=keep-id`
docker run \
    --user "$(id -u):$(id -g)" \
    --rm -ti \
    -v "$PWD/pc:/pc:rw" \
    -v "$PWD/twitchbot:/src:ro" \
    --workdir /src \
    ghcr.io/pre-commit-ci/runner-image \
    pre-commit install-hooks
docker run \
    --net=none \
    --user "$(id -u):$(id -g)" \
    --rm -ti \
    -v "$PWD/pc:/pc:ro" \
    -v "$PWD/twitchbot:/src:rw" \
    --workdir /src \
    ghcr.io/pre-commit-ci/runner-image \
    pre-commit run --all-files

$ bash t.sh 
+ rm -rf twitchbot pc
+ git clone git@github.com:ChicoCodes/twitchbot
Cloning into 'twitchbot'...
remote: Enumerating objects: 234, done.
remote: Counting objects: 100% (234/234), done.
remote: Compressing objects: 100% (141/141), done.
remote: Total 234 (delta 109), reused 172 (delta 65), pack-reused 0
Receiving objects: 100% (234/234), 65.71 KiB | 1.37 MiB/s, done.
Resolving deltas: 100% (109/109), done.
+ mkdir pc
++ id -u
++ id -g
+ docker run --userns=keep-id --user 1000:1000 --rm -ti -v /tmp/y/pc:/pc:rw -v /tmp/y/twitchbot:/src:ro --workdir /src ghcr.io/pre-commit-ci/runner-image pre-commit install-hooks
[INFO] Initializing environment for https://github.com/golangci/golangci-lint.
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/golangci/golangci-lint.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
++ id -u
++ id -g
+ docker run --userns=keep-id --net=none --user 1000:1000 --rm -ti -v /tmp/y/pc:/pc:ro -v /tmp/y/twitchbot:/src:rw --workdir /src ghcr.io/pre-commit-ci/runner-image pre-commit run --all-files
golangci-lint............................................................Failed
- hook id: golangci-lint
- exit code: 3

ERRO Running error: context loading failed: failed to load packages: failed to load with go/packages: err: exit status 1: stderr: go: github.com/gempir/go-twitch-irc/v2@v2.5.0: Get "https://proxy.golang.org/github.com/gempir/go-twitch-irc/v2/@v/v2.5.0.mod": dial tcp: lookup proxy.golang.org on [::1]:53: read udp [::1]:37117->[::1]:53: read: connection refused

Fix End of Files.........................................................Passed
Trim Trailing Whitespace.................................................Passed

I plan to add these instructions to the website at some point but haven't gotten around to it just yet

I do at some point want to have some "add-ons" which enable something like this, but I haven't figured out a way to do this safely and fast without blowing through a bunch of cost so it'll likely be a pro-tier feature

[fsouza]

Right, but that lib is a dependency of the project being linted, so it can't be linted without having dependencies available (something like gofmt is fine, but something that requires compiling the code as part of linting won't work - either way go will always try to fetch the dependencies on demand). How does it work with mypy-enabled projects? Do you have to run mypy with --ignore-missing-imports or list dependencies in additional_dependencies in addition to the language's standard location for dependency management?

[asottile]

yeah mypy works similarly (doesn't have access to dependencies) -- usually run with --ignore-missing-imports and then gradually add dependencies that have types into additional_dependencies

[fsouza]

Nice, thanks for the info! :D Sorry for opening a question-style issue, closing x)

[asottile]

nah this is fine -- I want to implement this eventually :)

[Mause]

This is also an issue for cargo check and cargo clippy for rust unfortunately

[asottile]

I've just released a brand new "lite" version of pre-commit.ci which may be of interest (you could install your repository-specific tooling before running the fixes in github actions). you can read more about it at https://pre-commit.ci/lite -- the tl;dr is adds autofixing to GitHub Actions