Merge remote-tracking branch 'origin/part4' into part4

Conflicts:
	README.md

README.md

@@ -7,6 +7,16 @@ For details about the use cases demonstrated in this project, please read the bl
 this is a work in progress, with dedicated branches for each article. The master branch contains the sources for the
 most recent article.
 
+The [initial article](http://wp.me/p1E7sK-mv) ([part1.md](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/master/part1.md))
+gives a thorough overview over the complete pipeline, while each of the following build steps will be described in its own article:
+
+* [build, publish](http://wp.me/p1E7sK-nG) ([part2.md](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/master/part2.md))
+* [e2e test](http://wp.me/p1E7sK-oI) ([part3.md](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/master/part3.md))
+* contract test
+* build image
+* deploy on dev
+* deploy on prod
+
 
 Comments, Questions, Feedback
 =============================

part1.md

@@ -0,0 +1,65 @@
+This series of posts will show you some aspects of our continuous deployment pipeline for one of our products. It is built, tested and deployed to our servers by using [Gradle](http://www.gradle.org/), while the application itself runs inside [Docker](https://www.docker.com/) containers.
+
+We want to show you how we use Gradle to implement a complete pipeline with minimal dependency on command line tools. We'll also describe how to perform rollouts to production without the need for shell scripts or even remote shell access, by using the [Docker remote API](https://docs.docker.com/reference/api/docker_remote_api/). All details regarding our [AngularJS](https://angularjs.org/) frontend, test concepts for multi-product compatibility and detailed code examples will be explained in upcoming posts, with [example code](https://github.com/gesellix/pipeline-with-gradle-and-docker) provided at GitHub. This post starts with a bird's-eye view of our pipeline.
+
+Overview
+=
+
+Our deployment pipeline is divided into six build goals, combined in a [TeamCity](http://www.jetbrains.com/teamcity/) *Build Chain*. We'll add links to each build goal as soon as a corresponding article has been published:
+
+* [build, publish](http://wp.me/p1E7sK-nG)
+* [e2e test](http://wp.me/p1E7sK-oI)
+* contract test
+* build image
+* deploy on dev
+* deploy on prod
+
+Every git push to a shared Git repository triggers a new build and is automatically deployed to production.
+
+The first step builds a multi module project and produces two Spring Boot jar files for our backend and frontend webapps. Both jars are published to our Nexus artifact repository. Building a [Spring Boot](http://projects.spring.io/spring-boot/) application with Gradle is straight-forward, you'll find examples in the Spring Boot [guides](http://spring.io/guides/gs/spring-boot/). The [gradle-grunt-plugin](http://plugins.gradle.org/plugin/com.moowork.grunt) helps us building and unit testing the AngularJS frontend by delegating build steps to the [Grunt](http://gruntjs.com/) task runner.
+
+Our *e2e-test* build step runs some integration tests on our frontend to ensure that it is compatible to our backend. The next step runs so-called *contract tests*, which runs cross-product tests to ensure our new release still plays well with the other services on our platform.
+
+The fourth step builds a Docker image containing both frontend and backend webapps and pushes it to a private Docker registry. After that, we pull the newly built image to our development and production stages and run container instances. In order to maximize product availability, both stages use [blue-green deployment](http://martinfowler.com/bliki/BlueGreenDeployment.html).
+
+Gradle and Groovy power
+=
+
+As already mentioned, the complete pipeline is implemented using Gradle. Running the build and publish tasks is quite trivial, some code snippets will be shown in the following posts. The integration of our frontend build using the gradle-grunt-plugin has been straight forward, too, while we added some configuration to let Gradle know about Grunt's inputs and outputs. That way, we enable Gradle to use its cache and [skip up to date tasks](http://www.gradle.org/docs/current/userguide/more_about_tasks.html#sec:up_to_date_checks) when there aren't any code changes.
+
+Running the e2e-tests and contract-tests wasn't possible with existing plugins, so we had to create some special tasks. Since Gradle lets us write native Groovy code, we didn't need to create dedicated shell scripts, but [execute commands](http://groovy.codehaus.org/Executing+External+Processes+From+Groovy) as simply as `"command".execute()`. That way we can perform the following steps to run our e2e-tests with [Protractor](http://www.protractortest.org):
+
+* start selenium-server
+* start e2e-reverse-proxy
+* start frontend and backend
+* run protractor e2e-tests
+* tear down
+
+In contrast to the e2e-tests, where we only check our frontend and backend application, we have some contract-tests to check our interaction with other services. Our backend interacts with some other products of our platform, and we want to be sure that after deploying a new release of our product, it still works together with current versions of the other products. Our contract-tests are implemented as [Spock framework](http://spockframework.org/) and [TestNG](http://testng.org/) tests and are a submodule of our product. A dedicated contract-tester module in an own project performs all necessary steps to find and run the external webapps in their released versions and to perform our contract-tests against their temporary instances. Like with the e2e-tests, all steps are implemented in Gradle, but this time we could use plugins like [Gradle Cargo plugin](https://github.com/bmuschko/gradle-cargo-plugin) and [Gradle Download Task](https://github.com/michel-kraemer/gradle-download-task), furthermore Gradle's built in test runner and dynamic dependency resolution for our contract-tests artifact:
+
+* collect participating product versions
+* download each product's webapp from Nexus
+* start the participating webapps and infrastructure services
+* run contract-tests
+* tear down
+
+Gradle and Docker
+=
+
+With our artifacts being tested, we package them in Docker images, deploy the images to our private registries and run fresh containers on our servers. Docker allows us to describe the image contents by writing [Dockerfiles](https://docs.docker.com/reference/builder/) as plain text, so that we can include all build instructions in our Git repository. Before using a [Gradle Docker plugin](http://plugins.gradle.org/plugin/de.gesellix.docker), we used Gradle to orchestrate Docker clients, which had to be installed on our TeamCity agents and the application servers. Like described above, we used the Groovy command executor to access the [Docker command line interface](https://docs.docker.com/reference/commandline/cli/). We're now in a transition to only use the Docker remote API, so that we don't need a Docker client on every build server, but only need to point the plugin to any Docker enabled server.
+
+Building and distributing our images, followed by starting the containers is only one part of our deployment. In order to implement continuous delivery without interrupting availability of our product, we implemented blue-green deployment. Therefore, our Gradle deployment script needs to ask our reverse proxy in front of our application servers for a deployable stage (e.g. green), perform the Docker container tasks and toggle a switch from the current to the new stage, e.g. from blue to green:
+
+* get the deployable stage
+* pull the new image from the Docker registry
+* stop and remove the old container
+* run a new container based on the new image
+* cleanup (e.g. remove unused images)
+* switch to the new stage with the fresh container
+
+Summary
+=
+
+With this brief overview you should have an impression of the key elements of our pipeline. In the upcoming posts we'll dive into each of these build steps, provide some code examples and discuss our experience regarding the chosen technologies and frameworks in context of our server setups.
+
+If you'd like to know special details, please leave a comment or contact us via Twitter [@gesellix](https://twitter.com/gesellix), so that we can include your wishes in the following posts. Even if you'd like us to talk about non technical aspects, e.g. like our experience introducing the above technologies to our teams, just ask!

part2.md

@@ -0,0 +1,59 @@
+After a quite long holiday break we now continue our series about the [Continuous Deployment Pipeline with Gradle and Docker](http://wp.me/p1E7sK-mv).
+
+This post is about the first step where our build chain creates the Spring Boot packages and publishes them to our Nexus repository manager. As shown in the high-level overview below, it is only a quite small part of the complete pipeline:
+<a href="https://hypoport.files.wordpress.com/2014/10/deployment-pipeline.png"><img class="aligncenter wp-image-1475 size-large" src="https://hypoport.files.wordpress.com/2014/10/deployment-pipeline.png?w=584" alt="Deployment Pipeline with Gradle and Docker" width="584" height="353" /></a>
+
+Gradle and Spring Boot provide you a very convenient build and plugin system and work out of the box for standard builds. Yet, the devil is in the details. Our project consists of a multi module setup with the following subprojects:
+
+* backend
+* frontend
+* common
+* contract-test
+* e2e-test
+
+The projects _backend_ and _frontend_ are our main modules with each being deployed as a standalone application. They share the _common_ project which contains the security and web config. The _contract-test_ and _e2e-test_ projects contain more integrative tests and will be discussed later in dedicated posts.
+
+We'll now take a deep dive into our build scripts and module structure. You can find the [example source code on GitHub](https://github.com/gesellix/pipeline-with-gradle-and-docker/tree/part2), where we provide a minimal, but working project with the important parts being described here.
+
+Gradle project setup
+=
+
+A build on our CI-Server TeamCity uses the [Gradle Wrapper](http://www.gradle.org/docs/current/userguide/gradle_wrapper.html) by running the tasks `build` and `publish`. These tasks are called on the root level of our project. Our Gradle root project contains the common configuration so that the subprojects only need to configure minimal aspects or special plugins.
+
+Shared dependency versions are defined in the root project, so that all subprojects use the same dependency versions. Gradle also allows you to define sets of dependencies, so that you can reference them as complete package without known its details. We call these sets _libraries_ and you can find an example at the root [build.gradle](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/build.gradle#L63) along with its usage in the [dependency closure](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/build.gradle#L109).
+
+Using a common definition of dependencies sometimes isn't enough, because you also have to handle transitive dependencies. You have the option to manage transitive dependencies by [manually excluding](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/build.gradle#L96) or even redefining them. Another option we often use is to override clashing dependency versions by configuring the build script's configuration. The `resolutionStrategy` can be configured to fail when version conflicts are recognized. The example project shows you how we globally [manage our dependencies](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/build.gradle#L118).
+
+Spring Boot configuration
+=
+
+Building a Spring Boot application with Gradle is simplified with the help of the [Spring Boot Gradle Plugin](http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#build-tool-plugins-gradle-plugin). The plugin configures your build script so that running `gradle build` depends on the `bootRepackage` task.
+
+You'll see in the backend and frontend _build.gradle_ scripts, that we configure Gradle to [replace a token](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/frontend/build.gradle#L72) in our source files with the `artifactVersion`. This special token replacement aims at setting the actual version in our _application.properties_ file, which is used to configure Spring Boot. By adding a line like `info.build.version=@example.version@` we enable the `/info` endpoint so that we can ask a running application about its version. The version will be used later in our deployment pipeline. Details on our artifact versioning scheme will be described in the section about publishing below.
+
+Performing Node.js build tasks
+=
+
+Our backend build isn't very spectacular, but our frontend build needs some more explanation. We implemented our frontend with AngularJS, but use Spring Boot to deliver the static resources and to implement security. Before packaging the AngularJS resources in the frontend artifact, we let Gradle perform a `grunt release` task. [Grunt](http://gruntjs.com/) is a [Node.js](http://nodejs.org/) based task runner, which lets us run unit tests, minimize our frontend code or even images and package everything. Its result then [needs to be copied](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/frontend/build.gradle#L67) to the _public_ resources folder of Spring Boot.
+
+Configuring a Node.js build in a platform neutral way isn't one of the trivial tasks, but we use the [gradle-grunt-plugin](https://plugins.gradle.org/plugin/com.moowork.grunt) and the [gradle-node-plugin](https://plugins.gradle.org/plugin/com.moowork.node) which helps a lot. Apart from delegating the grunt release to the plugin we also configure the according _grunt\_release_ task to recognize _inputs_ and _outputs_ in the Gradle [build script](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/frontend/build.gradle#L51). The _inputs_ and _outputs_ help Gradle to decide if the task needs to be executed. If there haven't been any source changes and the output still exists, the task is regarded up to date and will be skipped.
+
+Publishing and versioning Gradle artifacts
+=
+
+With both _frontend_ and _backend_ being packaged as artifacts, we would like to publish them to our Nexus artifact repository. Nexus needs the well known set of _groupId_, _artifactId_ and _version_ to identify an artifact. The Gradle `maven-publish` plugin can be configured in a very convenient way to use the project's group, name and version as Maven coordinates. As you can see in the example source code, we already [configure the group](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/build.gradle#L28) in our root project. The subproject's name fits our needs as artifactId, which leads us to the final property, the _version_.
+
+We wanted the version to be unique and sortable by the artifact's build time. We also didn't want to maintain a `version.txt` in our project. Long story short, we defined our version to look like the scheme: `yyyy-MM-dd'T'HH-mm-ss_git-commit-hash`. The part before the `_` corresponds to the build timestamp and the second part corresponds to the latest commit hash of the project's git repository. That way we can quickly recognize when the artifact has been build with which commit in the project's history.
+
+The artifact version is [generated on every build](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/build.gradle#L20). Apart from updating our `application.properties`, we also use the artifact version to [configure the `publish` task](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/build.gradle#L151) in our root project. The rest works out of the box, we only need to configure the Nexus [publish url with username and password](https://github.com/gesellix/pipeline-with-gradle-and-docker/blob/part2/build.gradle#L141).
+
+Build on a CI-Server
+=
+
+Our CI Server [TeamCity](https://www.jetbrains.com/teamcity/) now only needs to execute the `gradlew clean build publish` tasks to compile, perform all unit tests, package the Spring Boot applications and publish them to the artifact repository. That wouldn't be enough, because we also want to perform integration tests and deploy the applications to our internal and production stages.
+
+TeamCity provides a feature to declare so-called `build artifacts`, which can be used by subsequent build goals in our build chain. We want the other build goals to know the application version, so we write it into a text file on the build agent and pass it to all build goals in our pipeline. Every build goal then uses the version to fetch the artifact from Nexus. The image below shows all build goals of our build chain:
+
+<a href="https://hypoport.files.wordpress.com/2014/10/build-chain-prod.png"><img class="aligncenter size-medium wp-image-1498" src="https://hypoport.files.wordpress.com/2014/10/build-chain-prod.png?w=300" alt="Build Chain" width="300" height="61" /></a>
+
+The selected yellow box in the build chain corresponds to the build step we described in this article. As promised, the next article in our series will describe you in detail how we perform our integrative e2e- and contract-tests. Comments and feedback here or [@gesellix](https://twitter.com/gesellix) are welcome!