Skip to content

Update Dockerfile #5159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Mar 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
cf456ab
Update Dockerfile
mehrdadbn9 Feb 18, 2023
e6945e0
Merge branch 'cortexproject:master' into master
mehrdadbn9 Feb 21, 2023
e59c0bc
Use build image for Go 1.20.1
alvinlin123 Mar 2, 2023
da8c79f
Merge branch 'master' into master
alvinlin123 Mar 2, 2023
e0201fd
Fix 'detected dubious ownership' error from git
alvinlin123 Mar 2, 2023
9b4c855
opps, fix careless typo
alvinlin123 Mar 2, 2023
c86be3f
Remove settingup safe.direcotry for non-container jobs
alvinlin123 Mar 2, 2023
0d34ff0
Need to upgrade go4.org/intern for go 1.20
alvinlin123 Mar 2, 2023
35481f7
increase timeout for golangci-lint see if it pass the build
alvinlin123 Mar 3, 2023
3208866
increase timeout for golangci-lint see if it pass the build
alvinlin123 Mar 3, 2023
d624428
let's see if newever version of golanci-lint is faster
alvinlin123 Mar 3, 2023
fa82ca1
dummy commit
alvinlin123 Mar 3, 2023
4573215
remove dummy commit file
alvinlin123 Mar 3, 2023
a36d0a5
fix bogus space in workflow file
alvinlin123 Mar 3, 2023
483669e
Fix bunch of revive linter errors
alvinlin123 Mar 3, 2023
5430134
Update doc for updating Go runtime
alvinlin123 Mar 3, 2023
91bca4d
Update .github/workflows/test-build-deploy.yml
alvinlin123 Mar 3, 2023
b815b5d
Update .github/workflows/test-build-deploy.yml
alvinlin123 Mar 3, 2023
a397947
Update .github/workflows/test-build-deploy.yml
alvinlin123 Mar 3, 2023
a755b3b
Update .github/workflows/test-build-deploy.yml
alvinlin123 Mar 3, 2023
8cdba37
Update .github/workflows/test-build-deploy.yml
alvinlin123 Mar 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 38 additions & 13 deletions .github/workflows/test-build-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,15 @@ jobs:
lint:
runs-on: ubuntu-20.04
container:
image: quay.io/cortexproject/build-image:upgrade-to-go1.19.3-e8b98ddc0
image: quay.io/cortexproject/build-image:upgrade-to-go1.20.1-pr-5159
steps:
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Git safe.directory
run: |
echo "this step is needed because when running in container, actions/checkout does not set safe.directory effectively."
echo "See https://github.com/actions/runner/issues/2033. We should use --system instead of --global"
git config --system --add safe.directory $GITHUB_WORKSPACE
# Commands in the Makefile are hardcoded with an assumed file structure of the CI container
# Symlink ensures paths specified in previous commands don’t break
- name: Sym Link Expected Path to Workspace
Expand All @@ -35,10 +40,15 @@ jobs:
test:
runs-on: ubuntu-20.04
container:
image: quay.io/cortexproject/build-image:upgrade-to-go1.19.3-e8b98ddc0
image: quay.io/cortexproject/build-image:upgrade-to-go1.20.1-pr-5159
steps:
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Git safe.directory
run: |
echo "this step is needed because when running in container, actions/checkout does not set safe.directory effectively."
echo "See https://github.com/actions/runner/issues/2033. We should use --system instead of --global"
git config --system --add safe.directory $GITHUB_WORKSPACE
- name: Sym Link Expected Path to Workspace
run: |
mkdir -p /go/src/github.com/cortexproject/cortex
Expand All @@ -49,10 +59,15 @@ jobs:
build:
runs-on: ubuntu-20.04
container:
image: quay.io/cortexproject/build-image:upgrade-to-go1.19.3-e8b98ddc0
image: quay.io/cortexproject/build-image:upgrade-to-go1.20.1-pr-5159
steps:
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Git safe.directory
run: |
echo "this step is needed because when running in container, actions/checkout does not set safe.directory effectively."
echo "See https://github.com/actions/runner/issues/2033. We should use --system instead of --global"
git config --system --add safe.directory $GITHUB_WORKSPACE
- name: Install Docker Client
run: ./.github/workflows/scripts/install-docker.sh
- name: Sym Link Expected Path to Workspace
Expand Down Expand Up @@ -92,17 +107,17 @@ jobs:
fail-fast: false
matrix:
tags:
- requires_docker
- integration_alertmanager
- integration_backward_compatibility
- integration_memberlist
- integration_querier
- integration_ruler
- requires_docker
- integration_alertmanager
- integration_backward_compatibility
- integration_memberlist
- integration_querier
- integration_ruler
Comment on lines +110 to +115
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for fixing my linting bug :P. We should test lint on these yamls

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I didn't fix it ... my IDE does it for me ;)

steps:
- name: Upgrade golang
uses: actions/setup-go@v2
with:
go-version: 1.19.x
go-version: 1.20.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 More specific is better

- name: Checkout Repo
uses: actions/checkout@v2
- name: Install Docker Client
Expand Down Expand Up @@ -173,20 +188,25 @@ jobs:
run: |
touch build-image/.uptodate
MIGRATIONS_DIR=$(pwd)/cmd/cortex/migrations
make BUILD_IMAGE=quay.io/cortexproject/build-image:upgrade-to-go1.19.3-e8b98ddc0 TTY='' configs-integration-test
make BUILD_IMAGE=quay.io/cortexproject/build-image:upgrade-to-go1.20.1-pr-5159 TTY='' configs-integration-test

deploy_website:
needs: [build, test]
if: (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')) && github.repository == 'cortexproject/cortex'
runs-on: ubuntu-20.04
container:
image: quay.io/cortexproject/build-image:upgrade-to-go1.19.3-e8b98ddc0
image: quay.io/cortexproject/build-image:upgrade-to-go1.20.1-pr-5159
steps:
- name: Checkout Repo
uses: actions/checkout@v2
with:
# web-deploy script expects repo to be cloned with ssh for some commands to work
ssh-key: ${{ secrets.WEBSITE_DEPLOY_SSH_PRIVATE_KEY }}
- name: Setup Git safe.directory
run: |
echo "this step is needed because when running in container, actions/checkout does not set safe.directory effectively."
echo "See https://github.com/actions/runner/issues/2033. We should use --system instead of --global"
git config --system --add safe.directory $GITHUB_WORKSPACE
- name: Sym Link Expected Path to Workspace
run: |
mkdir -p /go/src/github.com/cortexproject/cortex
Expand Down Expand Up @@ -217,10 +237,15 @@ jobs:
if: (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')) && github.repository == 'cortexproject/cortex'
runs-on: ubuntu-20.04
container:
image: quay.io/cortexproject/build-image:upgrade-to-go1.19.3-e8b98ddc0
image: quay.io/cortexproject/build-image:upgrade-to-go1.20.1-pr-5159
steps:
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Git safe.directory
run: |
echo "this step is needed because when running in container, actions/checkout does not set safe.directory effectively."
echo "See https://github.com/actions/runner/issues/2033. We should use --system instead of --global"
git config --system --add safe.directory $GITHUB_WORKSPACE
- name: Install Docker Client
run: ./.github/workflows/scripts/install-docker.sh
- name: Sym link Expected Path to Workspace
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
* [ENHANCEMENT] Alertmanager: Added `-alertmanager.enabled-tenants` and `-alertmanager.disabled-tenants` to explicitly enable or disable alertmanager for specific tenants. #5116
* [ENHANCEMENT] Upgraded Docker base images to `alpine:3.17`. #5132
* [ENHANCEMENT] Add retry logic to S3 bucket client. #5135
* [ENHANCEMENT] Update Go version to 1.20.1. #5159
* [FEATURE] Querier/Query Frontend: support Prometheus /api/v1/status/buildinfo API. #4978
* [FEATURE] Ingester: Add active series to all_user_stats page. #4972
* [FEATURE] Ingester: Added `-blocks-storage.tsdb.head-chunks-write-queue-size` allowing to configure the size of the in-memory queue used before flushing chunks to the disk . #5000
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ build-image/$(UPTODATE): build-image/*
SUDO := $(shell docker info >/dev/null 2>&1 || echo "sudo -E")
BUILD_IN_CONTAINER := true
BUILD_IMAGE ?= $(IMAGE_PREFIX)build-image
LATEST_BUILD_IMAGE_TAG ?= upgrade-to-go1.19.3-e8b98ddc0
LATEST_BUILD_IMAGE_TAG ?= upgrade-to-go1.20.1-e6945e022

# TTY is parameterized to allow Google Cloud Builder to run builds,
# as it currently disallows TTY devices. This value needs to be overridden
Expand Down
4 changes: 2 additions & 2 deletions build-image/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.19.3-buster
FROM golang:1.20.1-buster
ARG goproxyValue
ENV GOPROXY=${goproxyValue}
RUN apt-get update && apt-get install -y curl python-requests python-yaml file jq unzip protobuf-compiler libprotobuf-dev && \
Expand All @@ -23,7 +23,7 @@ RUN GOARCH=$(go env GOARCH) && \
chmod +x shfmt && \
mv shfmt /usr/bin

RUN curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh| sh -s -- -b /usr/bin v1.48.0
RUN curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh| sh -s -- -b /usr/bin v1.51.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reason for upgrading the version here is because the older version times out while running linter. However, the new version flagged few errors which I fixed/suppressed the in this PR.


ENV HUGO_VERSION=v0.101.0
RUN go install github.com/client9/misspell/cmd/misspell@v0.3.4 &&\
Expand Down
1 change: 1 addition & 0 deletions docs/contributing/how-to-update-the-build-image.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,5 @@ The build image currently can only be updated by a Cortex maintainer. If you're
1. `docker login quay.io`. Note that pushing to `quay.io/cortexproject/build-image` repository can only be done by a maintainer.
1. Build the and publish the image by using `make push-multiarch-build-image`. This will build and push multiplatform docker image (for linux/amd64 and linux/arm64). Running this step successfully requires [Docker Buildx](https://docs.docker.com/buildx/working-with-buildx/), but does not require a specific platform.
1. Replace the image tag in `.github/workflows/*` (_there may be multiple references_) and Makefile (variable `LATEST_BUILD_IMAGE_TAG`).
1. If you are updating Go's runtime version be sure to change `actions/setup-go`'s `go-version` in ``.github/workflows/*`.
1. Open a PR and make sure the CI with new build-image passes
4 changes: 2 additions & 2 deletions docs/contributing/how-to-upgrade-golang-version.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ weight: 4
slug: how-to-upgrade-golang-version
---

To upgrade the Golang version:
To upgrade the Go's runtime version:

1. Upgrade build image version with golang version as describe [here](./how-to-update-the-build-image.md)

If the minimum support Golang version should be upgraded as well:
If the minimum support Go's language version should be upgraded as well:

1. Upgrade `go` version in `go.mod`
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,8 @@ require (
go.uber.org/goleak v1.2.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
go.uber.org/zap v1.21.0 // indirect
go4.org/intern v0.0.0-20220617035311-6925f38cc365 // indirect
go4.org/unsafe/assume-no-moving-gc v0.0.0-20220617031537-928513b29760 // indirect
go4.org/intern v0.0.0-20230205224052-192e9f60865c // indirect
go4.org/unsafe/assume-no-moving-gc v0.0.0-20230221090011-e4bae7ad2296 // indirect
golang.org/x/crypto v0.6.0 // indirect
golang.org/x/exp v0.0.0-20230124195608-d38c7dcee874 // indirect
golang.org/x/mod v0.7.0 // indirect
Expand Down
9 changes: 5 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1693,10 +1693,11 @@ go.uber.org/zap v1.13.0/go.mod h1:zwrFLgMcdUuIBviXEYEH1YKNaOBnKXsx2IPda5bBwHM=
go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=
go.uber.org/zap v1.21.0 h1:WefMeulhovoZ2sYXz7st6K0sLj7bBhpiFaud4r4zST8=
go.uber.org/zap v1.21.0/go.mod h1:wjWOCqI0f2ZZrJF/UufIOkiC8ii6tm1iqIsLo76RfJw=
go4.org/intern v0.0.0-20220617035311-6925f38cc365 h1:t9hFvR102YlOqU0fQn1wgwhNvSbHGBbbJxX9JKfU3l0=
go4.org/intern v0.0.0-20220617035311-6925f38cc365/go.mod h1:WXRv3p7T6gzt0CcJm43AAKdKVZmcQbwwC7EwquU5BZU=
go4.org/unsafe/assume-no-moving-gc v0.0.0-20220617031537-928513b29760 h1:FyBZqvoA/jbNzuAWLQE2kG820zMAkcilx6BMjGbL/E4=
go4.org/unsafe/assume-no-moving-gc v0.0.0-20220617031537-928513b29760/go.mod h1:FftLjUGFEDu5k8lt0ddY+HcrH/qU/0qk+H8j9/nTl3E=
go4.org/intern v0.0.0-20230205224052-192e9f60865c h1:b8WZ7Ja8nKegYxfwDLLwT00ZKv4lXAQrw8LYPK+cHSI=
go4.org/intern v0.0.0-20230205224052-192e9f60865c/go.mod h1:RJ0SVrOMpxLhgb5noIV+09zI1RsRlMsbUcSxpWHqbrE=
go4.org/unsafe/assume-no-moving-gc v0.0.0-20230204201903-c31fa085b70e/go.mod h1:FftLjUGFEDu5k8lt0ddY+HcrH/qU/0qk+H8j9/nTl3E=
go4.org/unsafe/assume-no-moving-gc v0.0.0-20230221090011-e4bae7ad2296 h1:QJ/xcIANMLApehfgPCHnfK1hZiaMmbaTVmPv7DAoTbo=
go4.org/unsafe/assume-no-moving-gc v0.0.0-20230221090011-e4bae7ad2296/go.mod h1:FftLjUGFEDu5k8lt0ddY+HcrH/qU/0qk+H8j9/nTl3E=
golang.org/x/crypto v0.0.0-20171113213409-9f005a07e0d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181009213950-7c1a557ab941/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
Expand Down
2 changes: 1 addition & 1 deletion integration/ruler_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ func TestRulerEvaluationDelay(t *testing.T) {
now := time.Now()

// Generate series that includes stale nans
var samplesToSend int = 10
samplesToSend := 10
series := prompb.TimeSeries{
Labels: []prompb.Label{
{Name: "__name__", Value: "a_sometimes_stale_nan_series"},
Expand Down
2 changes: 1 addition & 1 deletion pkg/ingester/instance_limits.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ type InstanceLimits struct {
}

// Sets default limit values for unmarshalling.
var defaultInstanceLimits *InstanceLimits = nil
var defaultInstanceLimits *InstanceLimits

// UnmarshalYAML implements the yaml.Unmarshaler interface. If give
func (l *InstanceLimits) UnmarshalYAML(unmarshal func(interface{}) error) error {
Expand Down
3 changes: 3 additions & 0 deletions pkg/querier/iterators/compatibe_chunk_iterator.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,13 @@ type compatibleChunksIterator struct {
err error
}

//revive:disable:unexported-return
func NewCompatibleChunksIterator(i compChunksIterator) *compatibleChunksIterator {
return &compatibleChunksIterator{it: i}
}

//revive:enable:unexported-return

func (c *compatibleChunksIterator) Next() chunkenc.ValueType {
if c.it.Next() {
return chunkenc.ValFloat
Expand Down
3 changes: 3 additions & 0 deletions pkg/ring/kv/dynamodb/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -281,10 +281,13 @@ type mockDynamodbClient struct {
mock.Mock
}

//revive:disable:unexported-return
func NewDynamodbClientMock() *mockDynamodbClient {
return &mockDynamodbClient{}
}

//revive:enable:unexported-return

func (m *mockDynamodbClient) List(context.Context, dynamodbKey) ([]string, float64, error) {
args := m.Called()
var err error
Expand Down
2 changes: 0 additions & 2 deletions pkg/ring/kv/dynamodb/dynamodb.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,6 @@ type dynamoDbClient interface {
}

type dynamodbKV struct {
dynamoDbClient

ddbClient dynamodbiface.DynamoDBAPI
logger log.Logger
tableName *string
Expand Down
2 changes: 1 addition & 1 deletion pkg/ruler/compat.go
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ func DefaultTenantManagerFactory(cfg Config, p Pusher, q storage.Queryable, engi
q = querier.NewErrorTranslateQueryableWithFn(q, WrapQueryableErrors)

return func(ctx context.Context, userID string, notifier *notifier.Manager, logger log.Logger, reg prometheus.Registerer) RulesManager {
var queryTime prometheus.Counter = nil
var queryTime prometheus.Counter
if rulerQuerySeconds != nil {
queryTime = rulerQuerySeconds.WithLabelValues(userID)
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/util/allowed_tenants_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ func TestAllowedTenants_Combination(t *testing.T) {
}

func TestAllowedTenants_Nil(t *testing.T) {
var a *AllowedTenants = nil
var a *AllowedTenants

// All tenants are allowed when using nil as allowed tenants.
require.True(t, a.IsAllowed("A"))
Expand Down
14 changes: 13 additions & 1 deletion vendor/go4.org/unsafe/assume-no-moving-gc/assume-no-moving-gc.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions vendor/go4.org/unsafe/assume-no-moving-gc/untested.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions vendor/modules.txt

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.