Rehauled build system and integration testing

- USE_CONTAINER allow to seamlessly run targets inside or outside containers
- all build calls have been harmonized, honoring the same env variables
- contributing doc has been streamlined according to that
- kill the distinction between remote and local docker builds
- got rid of some of the byzantine calls in various asorted scripts
- support for static build, debug builds, verbose

Signed-off-by: Olivier Gambier <olivier@docker.com>

.dockerignore

@@ -1,2 +1,4 @@
 docker-machine*
-.git
+*.log
+bin
+cover

.gitignore

@@ -2,3 +2,5 @@ docker-machine*
 *.log
 *.iml
 .idea/
+bin
+cover

.travis.yml

@@ -1,15 +1,13 @@
 language: go
-sudo: false
 go:
-- 1.4.2
 - 1.5
 install:
-- export GOPATH=${TRAVIS_BUILD_DIR}/Godeps/_workspace:$GOPATH
-- export PATH=${TRAVIS_BUILD_DIR}/Godeps/_workspace/bin:$PATH
-- go get -t -v ./...
-- go get golang.org/x/tools/cmd/cover
+- go get github.com/mattn/goveralls
 script:
-- script/validate-dco
-- script/validate-gofmt
-- go test -v -short ./...
-- script/generate-coverage report
+- make dco
+- make fmt
+# - make lint
+# - make vet
+- make test-short
+- make test-long
+- make coverage-send

CONTRIBUTING.md

@@ -2,6 +2,7 @@
 
 [![GoDoc](https://godoc.org/github.com/docker/machine?status.png)](https://godoc.org/github.com/docker/machine)
 [![Build Status](https://travis-ci.org/docker/machine.svg?branch=master)](https://travis-ci.org/docker/machine)
+[![Coverage Status](https://coveralls.io/repos/docker/machine/badge.svg?branch=upstream-master&service=github)](https://coveralls.io/github/docker/machine?branch=upstream-master)
 
 Want to hack on Machine? Awesome! Here are instructions to get you
 started.
@@ -13,108 +14,139 @@ Docker does things, you'll feel right at home.
 Otherwise, please read [Docker's contributions
 guidelines](https://github.com/docker/docker/blob/master/CONTRIBUTING.md).
 
-# Building
+# Building using docker
 
 The requirements to build Machine are:
 
-1. A running instance of Docker
+1. A running instance of Docker (or alternatively a golang 1.5 development environment)
 2. The `bash` shell
 3. [Make](https://www.gnu.org/software/make/)
 
-To build, run:
+Call `export USE_CONTAINER=true` to instruct the build system to use containers to build.
+If you want to build natively using golang instead, don't set this variable.
+
+## Building
+
+To build the docker-machine binary, simply run:
 
     $ make
 
-From the Machine repository's root.  Machine will run the build inside of a
-Docker container and the compiled binaries will appear in the project directory
-on the host.
+From the Machine repository's root. You will now find a `docker-machine`
+binary at the root of the project.
 
-By default, Machine will run a build which cross-compiles binaries for a variety
-of architectures and operating systems.  If you know that you are only compiling
-for a particular architecture and/or operating system, you can speed up the
-compile by setting the standard `GOOS` and/or `GOARCH` environment variables to
-deviate from the defaults.
+You may call:
 
-For instance, if you only want to compile for use on OS X with the x86_64 arch,
-run:
+    $ make clean
 
-    $ GOOS=darwin GOARCH=amd64 make
+to clean-up build results.
 
-If you have any questions we're in #docker-machine on Freenode.
+## Tests and validation
 
-## Remote Build
+To run basic validation (dco, fmt), and the project unit tests, call:
 
-You can use Machine to build Machine:
+    $ make test
 
-    $ GOOS=darwin GOARCH=amd64 make remote
+If you want more indepth validation (vet, lint), and all tests with race detection, call:
 
-The difference is this works even if you don't have local docker daemon, but
-instead a docker host that you [set up using Machine](https://github.com/docker/machine).
+    $ make validate
 
-## Unit Tests
+If you make a pull request, it is highly encouraged that you submit tests for
+the code that you have added or modified in the same pull request.
 
-To run the unit tests for the whole project, using the following script:
+## Code Coverage
 
-    $ script/test
+To generate an html code coverage report of the Machine codebase, run:
 
-This will run the unit tests inside of a container, so you don't have to worry
-about configuring your environment properly before doing so.
+    make coverage-serve
 
-To run the unit tests for only a specific subdirectory of the project, you can
-pass an argument to that script to specify which directory, e.g.:
+And navigate to http://localhost:8000 (hit `CTRL+C` to stop the server).
 
-    $ script/test ./drivers/amazonec2
+Alternatively, if you are building natively, you can simply run:
 
-If you make a pull request, it is highly encouraged that you submit tests for
-the code that you have added or modified in the same pull request.
+    make coverage-html
 
-## Code Coverage
+This will generate and open the report file:
 
-Machine includes a script to check for missing `*_test.go` files and to generate
-an [HTML-based representation of which code is covered by tests](http://blog.golang.org/cover#TOC_5.).
+![](/docs/img/coverage.png)
 
-To run the code coverage script, execute:
+## List of all targets
 
-```console
-$ ./script/coverage serve
-```
+### High-level targets
 
-You will see the results of the code coverage check as they come in.
+    make clean
+    make build
+    make test
+    make validate
 
-This will also generate the code coverage website and serve it from a container
-on port 8000.  By default, `/` will show you the source files from the base
-directory, and you can navigate to the coverage for any particular subdirectory
-of the Docker Machine repo's root by going to that path.  For instance, to see
-the coverage for the VirtualBox driver's package, browse to `/drivers/virtualbox`.
+### Build targets
 
-![](/docs/img/coverage.png)
+Build a single, native machine binary:
+
+    make build-simple
+
+Build for all supported oses and architectures (binaries will be in the `bin` project subfolder):
 
-You can hit `CTRL+C` to stop the server.
+    make build-x
+
+Build for a specific list of oses and architectures:
+
+    TARGET_OS=linux TARGET_ARCH="amd64 arm" make build-x
+
+You can further control build options through the following environment variables:
+
+    DEBUG=true # enable debug build
+    STATIC=true # build static (note: when cross-compiling, the build is always static)
+    VERBOSE=true # verbose output
+    PARALLEL=X # lets you control build parallelism when cross-compiling multiple builds
+    PREFIX=folder
+
+Scrub build results:
+
+    make build-clean
+
+### Coverage targets
+
+    make coverage-html
+    make coverage-serve
+    make coverage-send
+    make coverage-generate
+    make coverage-clean
+
+### Tests targets
+
+    make test-short
+    make test-long
+    make test-integration
+
+### Validation targets
+
+    make fmt
+    make vet
+    make lint
+    make dco
 
 ## Integration Tests
 
 ### Setup
 
-We utilize [BATS](https://github.com/sstephenson/bats) for integration testing.
-This runs tests against the generated binary.  To use, first make sure to
-[install BATS](https://github.com/sstephenson/bats).  Then run `./script/build`
-to generate the binary for your system.
+We use [BATS](https://github.com/sstephenson/bats) for integration testing, so,
+first make sure to [install it](https://github.com/sstephenson/bats#installing-bats-from-source).
 
 ### Basic Usage
 
-Once you have the binary, the integration tests can be invoked using the
-`test/integration/run-bats.sh` wrapper script.
+Integration tests can be invoked calling `make test-integration`.
 
-Using this wrapper script, you can invoke a test or subset of tests for a
-particular driver.  To set the driver, use the `DRIVER` environment variable.
+:warn: you cannot run integration test inside a container for now.
+Be sure to unset the `USE_CONTAINER` env variable if you set it earlier, or alternatively
+call directly `./test/integration/run-bats.sh` instead of `make test-integration`.
 
-The following examples are all shown relative to the project's root directory,
-but you should be able to invoke them from any directory without issue.
+You can invoke a test or subset of tests for a particular driver.
+To set the driver, use the `DRIVER` environment variable.
 
 To invoke just one test:
 
 ```console
-$ DRIVER=virtualbox ./test/integration/run-bats.sh test/integration/core/core-commands.bats
+$ DRIVER=virtualbox make test-integration test/integration/core/core-commands.bats
  ✓ virtualbox: machine should not exist
  ✓ virtualbox: create
  ✓ virtualbox: ls
@@ -141,23 +173,17 @@ Successfully removed bats-virtualbox-test
 To invoke a shared test with a different driver:
 
 ```console
-$ DRIVER=digitalocean ./test/integration/run-bats.sh test/integration/core/core-commands.bats
+$ DRIVER=digitalocean make test-integration test/integration/core/core-commands.bats
 ...
 ```
 
 To invoke a directory of tests recursively:
 
 ```console
-$ DRIVER=virtualbox ./test/integration/run-bats.sh test/integration/core/
+$ DRIVER=virtualbox make test-integration test/integration/core/
 ...
 ```
 
-If you want to invoke a group of tests across two or more different drivers at
-once (e.g. every test in the `drivers` directory), at the time of writing there
-is no first-class support to do so - you will have to write your own wrapper
-scripts, bash loops, etc.  However, in the future, this may gain first-class
-support as usage patterns become more clear.
-
 ### Extra Create Arguments
 
 In some cases, for instance to test the creation of a specific base OS (e.g.
@@ -175,7 +201,7 @@ $ DRIVER=amazonec2 \
   AWS_ACCESS_KEY_ID=zzzzzzzzzzzzzzzz \
   AWS_AMI=ami-12663b7a \
   AWS_SSH_USER=ec2-user \
-  ./test/integration/run-bats.sh test/integration/core
+  make test-integration test/integration/core
 ```
 
 in order to run the core tests on Red Hat Enterprise Linux on Amazon.

Dockerfile

@@ -1,15 +1,11 @@
 FROM golang:1.5
 
-# TODO: Vendor these `go get` commands using Godep.
-RUN go get github.com/mitchellh/gox
-RUN go get github.com/aktau/github-release
-RUN go get github.com/tools/godep
-RUN go get golang.org/x/tools/cmd/cover
-
-ENV GOPATH /go/src/github.com/docker/machine/Godeps/_workspace:/go
-ENV MACHINE_BINARY /go/src/github.com/docker/machine/docker-machine
-ENV USER root
+RUN go get  github.com/mitchellh/gox \
+            github.com/golang/lint/golint \
+            github.com/mattn/goveralls \
+            golang.org/x/tools/cover \
+            github.com/aktau/github-release
 
 WORKDIR /go/src/github.com/docker/machine
 
-ADD . /go/src/github.com/docker/machine
+ADD . /go/src/github.com/docker/machine

Makefile

@@ -1,26 +1,31 @@
-.PHONY: test validate-dco validate-gofmt
-
-default: build
-
-remote: build-remote
-
-test:
-	script/test
-
-validate-dco:
-	script/validate-dco
-
-validate-gofmt:
-	script/validate-gofmt
-
-validate: validate-dco validate-gofmt test
-
-build: clean
-	script/build
-
-build-remote: clean
-	script/build-remote
-
-clean:
-	rm -f docker-machine_*
-	rm -rf Godeps/_workspace/pkg
+# Plain make targets if not requested inside a container
+ifeq ($(USE_CONTAINER),)
+	include Makefile.inc
+	include mk/main.mk
+else
+# Otherwise, with docker, swallow all targets and forward into a container
+DOCKER_IMAGE_NAME := "docker-machine-build"
+DOCKER_CONTAINER_NAME := "docker-machine-build-container"
+
+%:
+		@docker build -t $(DOCKER_IMAGE_NAME) .
+
+		@test -z '$(shell docker ps -a | grep $(DOCKER_CONTAINER_NAME))' || docker rm -f $(DOCKER_CONTAINER_NAME)
+
+		@docker run --name $(DOCKER_CONTAINER_NAME) \
+		    -e DEBUG \
+		    -e STATIC \
+		    -e VERBOSE \
+		    -e BUILDTAGS \
+		    -e PARALLEL \
+		    -e COVERAGE_DIR \
+		    -e TARGET_OS \
+		    -e TARGET_ARCH \
+		    -e PREFIX \
+		    $(DOCKER_IMAGE_NAME) \
+		    make $@
+
+		@test ! -d bin || rm -Rf bin
+		@test -z "$(findstring build,$@)" || docker cp $(DOCKER_CONTAINER_NAME):/go/src/github.com/docker/machine/bin bin
+
+endif

Makefile.inc

@@ -0,0 +1,39 @@
+# Project name, used to name the binaries
+PKG_NAME := docker-machine
+GH_USER ?= docker
+GH_REPO ?= machine
+
+# If true, disable optimizations and does NOT strip the binary
+DEBUG ?=
+# If true, "build" will produce a static binary (cross compile always produce static build regardless)
+STATIC ?=
+# If true, turn on verbose output for build
+VERBOSE ?=
+# Build tags
+BUILDTAGS ?=
+# Adjust parallelism for gox
+PARALLEL ?= -1
+# Coverage default directory
+COVERAGE_DIR ?= cover
+# Whether to perform targets inside a docker container, or natively on the host
+USE_CONTAINER ?=
+
+# List of cross compilation targets
+ifeq ($(TARGET_OS),)
+  TARGET_OS := darwin freebsd linux windows
+endif
+
+ifeq ($(TARGET_ARCH),)
+  TARGET_ARCH := amd64 arm 386
+endif
+
+# Output prefix, defaults to local directory if not specified
+ifeq ($(PREFIX),)
+  PREFIX := $(shell pwd)
+endif
+
+default: build
+clean: coverage-clean build-clean
+build: build-simple
+test: dco fmt test-short
+validate: dco fmt vet lint test-short test-long