Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introducing contrib builds #33

Merged
merged 1 commit into from
Nov 8, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@
# VS Code
.vscode

# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
bin/

# Emacs
*~
\#*\#
Expand Down
166 changes: 157 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,14 +1,132 @@
EXPORTERS := $(wildcard exporter/*/.)
RECEIVERS := $(wildcard receiver/*/.)
# More exclusions can be added similar with: -not -path './testbed/*'
ALL_SRC := $(shell find . -name '*.go' \
-not -path './testbed/*' \
-type f | sort)

.DEFAULT_GOAL := all
# All source code and documents. Used in spell check.
ALL_SRC_AND_DOC := $(shell find . \( -name "*.md" -o -name "*.go" -o -name "*.yaml" \) \
-type f | sort)

.PHONY: all $(EXPORTERS) $(RECEIVERS)
all: $(EXPORTERS) $(RECEIVERS)
$(EXPORTERS):
$(MAKE) -C $@
$(RECEIVERS):
$(MAKE) -C $@
# ALL_PKGS is used with 'go cover'
ALL_PKGS := $(shell go list $(sort $(dir $(ALL_SRC))))

GOTEST_OPT?= -race -timeout 30s
GOTEST_OPT_WITH_COVERAGE = $(GOTEST_OPT) -coverprofile=coverage.txt -covermode=atomic
GOTEST=go test
GOFMT=gofmt
GOIMPORTS=goimports
GOLINT=golint
GOVET=go vet
GOOS=$(shell go env GOOS)
ADDLICENCESE= addlicense
MISSPELL=misspell -error
MISSPELL_CORRECTION=misspell -w
STATICCHECK=staticcheck

GIT_SHA=$(shell git rev-parse --short HEAD)
BUILD_INFO_IMPORT_PATH=github.com/open-telemetry/opentelemetry-collector-contrib/internal/version
BUILD_X1=-X $(BUILD_INFO_IMPORT_PATH).GitHash=$(GIT_SHA)
ifdef VERSION
BUILD_X2=-X $(BUILD_INFO_IMPORT_PATH).Version=$(VERSION)
endif
BUILD_INFO=-ldflags "${BUILD_X1} ${BUILD_X2}"

all-pkgs:
@echo $(ALL_PKGS) | tr ' ' '\n' | sort

all-srcs:
@echo $(ALL_SRC) | tr ' ' '\n' | sort

.DEFAULT_GOAL := addlicense-fmt-vet-lint-goimports-misspell-staticcheck-test

.PHONY: all
all: addlicense-fmt-vet-lint-goimports-misspell-staticcheck-test otelcontribcol

.PHONY: addlicense-fmt-vet-lint-goimports-misspell-staticcheck-test
addlicense-fmt-vet-lint-goimports-misspell-staticcheck-test: addlicense fmt vet lint goimports misspell staticcheck test

.PHONY: test
test:
$(GOTEST) $(GOTEST_OPT) $(ALL_PKGS)

.PHONY: benchmark
benchmark:
$(GOTEST) -bench=. -run=notests $(ALL_PKGS)

.PHONY: ci
ci: fmt vet lint goimports misspell staticcheck test-with-cover otelcontribcol
$(MAKE) -C testbed install-tools
$(MAKE) -C testbed runtests

.PHONY: test-with-cover
test-with-cover:
@echo Verifying that all packages have test files to count in coverage
@scripts/check-test-files.sh $(subst github.com/open-telemetry/opentelemetry-collector-contrib/,./,$(ALL_PKGS))
@echo pre-compiling tests
@time go test -i $(ALL_PKGS)
$(GOTEST) $(GOTEST_OPT_WITH_COVERAGE) $(ALL_PKGS)
go tool cover -html=coverage.txt -o coverage.html

.PHONY: addlicense
addlicense:
@ADDLICENCESEOUT=`$(ADDLICENCESE) -y 2019 -c 'OpenTelemetry Authors' $(ALL_SRC) 2>&1`; \
if [ "$$ADDLICENCESEOUT" ]; then \
echo "$(ADDLICENCESE) FAILED => add License errors:\n"; \
echo "$$ADDLICENCESEOUT\n"; \
exit 1; \
else \
echo "Add License finished successfully"; \
fi

.PHONY: fmt
fmt:
@FMTOUT=`$(GOFMT) -s -l $(ALL_SRC) 2>&1`; \
if [ "$$FMTOUT" ]; then \
echo "$(GOFMT) FAILED => gofmt the following files:\n"; \
echo "$$FMTOUT\n"; \
exit 1; \
else \
echo "Fmt finished successfully"; \
fi

.PHONY: lint
lint:
@LINTOUT=`$(GOLINT) $(ALL_PKGS) 2>&1`; \
if [ "$$LINTOUT" ]; then \
echo "$(GOLINT) FAILED => clean the following lint errors:\n"; \
echo "$$LINTOUT\n"; \
exit 1; \
else \
echo "Lint finished successfully"; \
fi

.PHONY: goimports
goimports:
@IMPORTSOUT=`$(GOIMPORTS) -local github.com/open-telemetry/opentelemetry-collector-contrib -d . 2>&1`; \
if [ "$$IMPORTSOUT" ]; then \
echo "$(GOIMPORTS) FAILED => fix the following goimports errors:\n"; \
echo "$$IMPORTSOUT\n"; \
exit 1; \
else \
echo "Goimports finished successfully"; \
fi

.PHONY: misspell
misspell:
$(MISSPELL) $(ALL_SRC_AND_DOC)

.PHONY: misspell-correction
misspell-correction:
$(MISSPELL_CORRECTION) $(ALL_SRC_AND_DOC)

.PHONY: staticcheck
staticcheck:
$(STATICCHECK) ./...

.PHONY: vet
vet:
@$(GOVET) ./...
@echo "Vet finished successfully"

.PHONY: install-tools
install-tools:
Expand All @@ -18,3 +136,33 @@ install-tools:
golang.org/x/tools/cmd/goimports \
github.com/client9/misspell/cmd/misspell \
honnef.co/go/tools/cmd/staticcheck

.PHONY: otelcontribcol
otelcontribcol:
GO111MODULE=on CGO_ENABLED=0 go build -o ./bin/$(GOOS)/otelcontribcol $(BUILD_INFO) ./cmd/otelcontribcol

.PHONY: docker-component # Not intended to be used directly
docker-component: check-component
GOOS=linux $(MAKE) $(COMPONENT)
cp ./bin/linux/$(COMPONENT) ./cmd/$(COMPONENT)/
docker build -t $(COMPONENT) ./cmd/$(COMPONENT)/
rm ./cmd/$(COMPONENT)/$(COMPONENT)

.PHONY: check-component
check-component:
ifndef COMPONENT
$(error COMPONENT variable was not defined)
endif

.PHONY: docker-otelcontribcol
docker-otelcontribcol:
COMPONENT=otelcontribcol $(MAKE) docker-component

.PHONY: binaries
binaries: otelcontribcol

.PHONY: binaries-all-sys
binaries-all-sys:
GOOS=darwin $(MAKE) binaries
GOOS=linux $(MAKE) binaries
GOOS=windows $(MAKE) binaries
8 changes: 8 additions & 0 deletions cmd/otelcontribcol/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM alpine:latest as certs
RUN apk --update add ca-certificates

FROM scratch
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY otelcontribcol /
ENTRYPOINT ["/otelcontribcol"]
EXPOSE 55678 55679
54 changes: 54 additions & 0 deletions cmd/otelcontribcol/components.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
// Copyright 2019 OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
"github.com/open-telemetry/opentelemetry-collector/config"
"github.com/open-telemetry/opentelemetry-collector/defaults"
"github.com/open-telemetry/opentelemetry-collector/exporter"
"github.com/open-telemetry/opentelemetry-collector/oterr"
"github.com/open-telemetry/opentelemetry-collector/receiver"

"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/stackdriverexporter"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zipkinscribereceiver"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Both of these components have tests for default config and for factory. This should be our minimum bar for accepting components into the build. We can think about having stricter requirements but I believe that's bare minimum.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #34 to make such requirements explicit.

)

func components() (config.Factories, error) {
errs := []error{}
factories, err := defaults.Components()
if err != nil {
return config.Factories{}, err
tigrannajaryan marked this conversation as resolved.
Show resolved Hide resolved
}

receivers := []receiver.Factory{&zipkinscribereceiver.Factory{}}
for _, rcv := range factories.Receivers {
receivers = append(receivers, rcv)
}
factories.Receivers, err = receiver.Build(receivers...)
if err != nil {
errs = append(errs, err)
}

exporters := []exporter.Factory{&stackdriverexporter.Factory{}}
for _, exp := range factories.Exporters {
exporters = append(exporters, exp)
}
factories.Exporters, err = exporter.Build(exporters...)
if err != nil {
errs = append(errs, err)
}

return factories, oterr.CombineErrors(errs)
}
49 changes: 49 additions & 0 deletions cmd/otelcontribcol/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
// Copyright 2019 OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Program otelcontribcol is the Omnition Telemetry Service built on top of
// OpenTelemetry Service.
package main

import (
"log"

"github.com/open-telemetry/opentelemetry-collector/service"

"github.com/open-telemetry/opentelemetry-collector-contrib/internal/version"
)

func main() {
handleErr := func(err error) {
if err != nil {
log.Fatalf("Failed to run the service: %v", err)
}
}

factories, err := components()
handleErr(err)

info := service.ApplicationStartInfo{
ExeName: "otelcontribcol",
LongName: "OpenTelemetry Contrib Collector",
Version: version.Version,
GitHash: version.GitHash,
}

svc, err := service.New(factories, info)
handleErr(err)

err = svc.Start()
handleErr(err)
}
3 changes: 3 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ go 1.12
require (
github.com/client9/misspell v0.3.4
github.com/google/addlicense v0.0.0-20190907113143-be125746c2c4
github.com/open-telemetry/opentelemetry-collector v0.2.1-0.20191016224815-dfabfb0c1d1e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have this module together with main.go? This will follow the same pattern that we have for individual components.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I was planning on doing it if/when we introduce multiple builds from this repo (protobuf vs gogoproto) but will look into doing it right away. We'll have to keep this go.mod around anyway so this repo can be consumed as a lib easily (stuff other than explicit modules like receivers, exporters).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I think we should punt it for now. If we make it a module, it'll not be able to access any internal package from the rest of the repo. The cmd module will by default import code from Github master instead of the local code so local development will be annoying. We can add permanent replace directives but I think there will be more issues. We'll mostly need to update the Makefile, CI and some scritps to supports it. Added a ticket to track it: #35

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

github.com/open-telemetry/opentelemetry-collector-contrib/exporter/stackdriverexporter v0.0.0-20191021165924-bb954188ac10
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zipkinscribereceiver v0.0.0-20191021165924-bb954188ac10
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac
golang.org/x/tools v0.0.0-20190917162342-3b4f30a44f3b
honnef.co/go/tools v0.0.1-2019.2.3
Expand Down
Loading