docs: restructure the documentation (#253)

This PR restructures the documentation to make it more clear to the
reader where to find relevant information.

Signed-off-by: Matthew Fisher <matt.fisher@microsoft.com>

README.md

@@ -3,39 +3,32 @@
 [![Coverage Status](https://codecov.io/gh/Azure/aks-engine/branch/master/graph/badge.svg)](https://codecov.io/gh/Azure/aks-engine)
 [![GoDoc](https://godoc.org/github.com/Azure/aks-engine?status.svg)](https://godoc.org/github.com/Azure/aks-engine)
 
-## Overview
-
-AKS-Engine leverages ARM (Azure Resource Manager) to build Kubernetes IaaS in Azure. AKS-Engine provides convenient tooling to quickly bootstrap clusters, and implements cluster provisioning and lifecycle operations for [AKS](https://docs.microsoft.com/en-us/azure/aks), Azure's managed Kubernetes service offering.
-
-More info, including a thorough walkthrough is [here](docs/aksengine.md).
+AKS-Engine is the easiest way to provision a self-managed Kubernetes cluster on Azure.
 
-Please see the [FAQ](/docs/faq.md) for answers about AKS-Engine and its progenitor ACS-Engine.
-
-## User guides
+## Overview
 
-[This guide](docs/kubernetes.md) walks you through your first cluster deployment.
+AKS-Engine provides convenient tooling to quickly bootstrap Kubernetes clusters on Azure. By leveraging [ARM (Azure Resource Manager)][ARM], AKS-Engine helps you create, destroy and maintain clusters provisioned with basic IaaS resources in Azure.
 
-These guides cover more advanced features to try out after you have built your first cluster:
+## Getting started
 
-* [Cluster Definition](docs/clusterdefinition.md) - describes the components of the cluster definition file
-* [Custom VNET](examples/vnet) - shows how to use a custom VNET
-* [Attached Disks](examples/disks-storageaccount) - shows how to attach up to 4 disks per node
-* [Managed Disks](examples/disks-managed) - shows how to use managed disks
-* [Large Clusters](examples/largeclusters) - shows how to create cluster sizes of up to 1200 nodes
+Depending on how new you are to AKS-Engine, you can try [a tutorial][tutorials], or just dive straight into the [documentation][docs].
 
-## Contributing
+Please see the [FAQ][] for answers about AKS-Engine and its progenitor ACS-Engine.
 
-Follow the [developers guide](docs/developers.md) to set up your environment.
+## Sharpen your skills
 
-To build aks-engine, run `make build`. If you are developing with a working [Docker environment](https://docs.docker.com/engine), you can also run `make dev` (or `makedev.ps1` on Windows) first to start a Docker container and run `make build` inside the container.
+The official [AKS-Engine documentation][docs] covers everything you need to know about AKS-Engine (and then some).
 
-Please follow these instructions before submitting a PR:
+## Join the community
 
-1. Execute `make test` to run unit tests.
-2. Manually test deployments if you are making modifications to the templates.
-* For example, if you have to change the expected resulting templates then you should deploy the relevant example cluster definitions to ensure that you are not introducing any regressions.
-3. Make sure that your changes are properly documented and include relevant unit tests.
+Want to get involved? The [community guide][community] covers everything you need to know about the AKS-Engine community and how you can contribute.
 
 ## Code of conduct
 
-This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
+This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information, see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
+
+[ARM]: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview
+[community]: docs/community/README.md
+[docs]: docs/README.md
+[FAQ]: docs/faq.md
+[tutorials]: docs/tutorials/README.md

docs/README.md

@@ -1,20 +1,23 @@
-# AKS-Engine - Builds Kubernetes Clusters
-
-## Overview
-
-This cluster definition examples demonstrate how to create a customized Docker Enabled Cluster on Microsoft Azure.
-
-## User Guides
-
-* [AKS Engine](aksengine.md) - shows you how to build and use the AKS engine to generate custom Docker enabled container clusters
-* [Cluster Definition](clusterdefinition.md) - describes the components of the cluster definition file
-* [Kubernetes Walkthrough](kubernetes.md) - shows how to create a Kubernetes enabled Docker cluster on Azure
-* [Kubernetes Windows Walkthrough](kubernetes/windows.md) - shows how to create a hybrid Kubernetes Windows enabled Docker cluster on Azure.
-* [Kubernetes with GPU support Walkthrough](kubernetes/gpu.md) - shows how to create a Kubernetes cluster with GPU support.
-* [Kubernetes AAD integration Walkthrough](kubernetes/aad.md) - shows how to create a Kubernetes cluster with AAD as authentication provider.
-* [Kubernetes Monitoring Walkthrough](kubernetes/monitoring.md) - shows how to set up monitoring of your Kubernetes cluster
-* [Custom VNET](../examples/vnet) - shows how to use a custom VNET
-* [Attached Disks](../examples/disks-storageaccount) - shows how to attach up to 4 disks per node
-* [Managed Disks](../examples/disks-managed) (under private preview) - shows how to use managed disks
-* [Large Clusters](../examples/largeclusters) - shows how to create cluster sizes of up to 1200 nodes
-* [Running Kubernetes in a hybrid environment](kubernetes/hybrid-environment.md) - considerations around how you will configure your cluster, and plan your deployment when running Kubernetes in a Cloud/On-Premise environment
+# AKS-Engine documentation
+
+Everything you need to know about AKS-Engine.
+
+## How the documentation is organized
+
+AKS-Engine has a lot of documentation. A high-level overview of how it’s organized will help you know where to look for certain things.
+
+[Tutorials][] take you by the hand through a series of steps on a given subject. Start here if you’re new to AKS-Engine.
+
+[Topic guides][] discuss key topics and concepts at a fairly high level and provide useful background information and explanation.
+
+[How-to guides][] are recipes. They guide you through the steps involved in addressing key problems and use-cases. They are more advanced than tutorials and assume some knowledge of how AKS-Engine works.
+
+[Community guides][] teach you about the AKS-Engine community. It incudes information on the project's Code of Conduct, the planning process for the AKS-Engine project itself, its release cycle, and how you can contribute to the project.
+
+[quickstart]: tutorials/quickstart.md
+[getting started]: tutorials/hello-world.md
+[tutorials]: tutorials/README.md
+[How-to guides]: how-to/README.md
+[Topic guides]: topics/README.md
+[the reference guides]: reference/README.md
+[community guides]: community/README.md

docs/community/README.md

@@ -0,0 +1,14 @@
+# Community
+
+Here you'll find documentation geared towards learning about the development process for the AKS-Engine project itself and how you can contribute.
+
+- [Code of Conduct](code-of-conduct.md)
+- [Developer Guide](developer-guide.md)
+- [Planning Process](planning-process.md)
+- [Release Checklist](release-checklist.md)
+
+AKS-Engine is a community effort. As it keeps growing, we always need more people to help others. As soon as you learn AKS-Engine, you can contribute in many ways:
+
+- Join the #sig-azure Slack channel on <https://kubernetes.slack.com> and answer questions. By explaining AKS-Engine to other users, you’re going to learn a lot about the tool yourself.
+- Blog about AKS-Engine. We syndicate all the AKS-Engine blogs we know about on the [topics page](../topics/README.md); if you’d like to see your blog on that page, you are more than welcome to add it there.
+- Contribute to other projects that use AKS-Engine, write documentation, or release your own code as an open-source extension. The ecosystem of extensions is a community effort; help us build it!

docs/community/code-of-conduct.md

@@ -0,0 +1,3 @@
+## Code of conduct
+
+This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information, see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

docs/developers.md → docs/community/developer-guide.md

@@ -1,4 +1,4 @@
-# Developers Guide
+# Developer Guide
 
 This guide explains how to set up your environment for developing on
 aks-engine.
@@ -134,3 +134,19 @@ Example launch.json file:
   ]
 }
 ```
+
+## Test pipeline
+
+AKS-Engine employs a Continuous Integration (CI) system that incorporates Azure DevOps, configured to interact with the AKS-Engine GitHub project.
+
+The following steps constitute the AKS-Engine CI pipeline:
+
+1. Contributor opens a Pull Request (PR) against the AKS-Engine project
+1. The PR triggers an Azure DevOps job that
+   + applies the changes to the HEAD of the master branch
+   + runs unit tests and code coverage reports
+   + generates multiple ARM templates for different deployment scenarios
+   + simultaneously provisions the clusters based on generated templates in Azure
+1. The PR is code reviewed by the members of AKS-Engine team
+1. Once the PR is approved and the end-to-end job has passed, the PR can now be merged into the master branch
+1. Once merged, another job is triggered to verify integrity of the master branch. This job is similar to the PR job.

docs/howto/README.md

@@ -0,0 +1,8 @@
+# "How-to" guides
+
+Here you’ll find short answers to “How do I….?” types of questions. These how-to guides don’t cover topics in depth – you’ll find that material in the [Topic Guides](../topics/). However, these guides will help you quickly accomplish common tasks.
+
+- [Troubleshooting](troubleshooting.md)
+- [Building Windows Kubernetes Binaries](building-windows-kubernetes-binaries.md)
+- [Large Kubernetes Clusters with AKS-Engine](kubernetes-large-clusters.md)
+- [Using HTTP Ingress Routing in a Mixed Cluster](mixed-cluster-ingress.md)

docs/kubernetes-build-win-binaries.md → docs/howto/building-windows-kubernetes-binaries.md

@@ -1,12 +1,15 @@
-# Building Windows Kubernetes Binaries and deploy to an Azure storage account
+# Building Windows Kubernetes Binaries
 
 ## Background
+
 Microsoft maintains a fork of the Kubernetes project at https://github.com/Azure/kubernetes which includes patches not yet included in upstream Kubernetes for release 1.7 and 1.8; these are needed for Windows containers to function. *From release 1.9, all Windows features are in upstream and Windows binaries no longer needs to be built from Azure fork.*
 
 ## Instructions
+
 The following instructions show how to deploy the Windows Kubernetes Binaries and deploy them to an Azure Storage Account.
 
 ### Prerequisites
+
 * Azure Storage Account and Azure Storage Container to store Windows binaries
 * Access to [wincni.exe] and [hns.psm1] (https://github.com/Microsoft/SDN/tree/master/Kubernetes/windows/). Windows CNI is a plugin that supports the Container Network Interface (CNI) network model and interfaces with the Windows Host Networking Service (HNS) to configure host networking and policy.
 * Docker installed and running. MacOS users using Docker for Mac must have at [least 3GB of memory allocated to Docker](https://github.com/kubernetes/kubernetes/tree/master/build/#requirements) or building will likely fail.

docs/kubernetes-large-clusters.md → docs/howto/kubernetes-large-clusters.md

@@ -1,15 +1,18 @@
-# Large Kubernetes Clusters with aks-engine
+# Large Kubernetes Clusters with AKS-Engine
 
 ## Background
-Starting from aks-engine v0.3.0, aks-engine supports using exponential cloud backoff that is a feature of Kubernetes v1.6.6 and newer. Cloud backoff allows Kubernetes nodes to backoff on HTTP 429 errors that are usually caused by exceeding Azure API limits.
+
+Starting from AKS-Engine v0.3.0, AKS-Engine supports using exponential cloud backoff that is a feature of Kubernetes v1.6.6 and newer. Cloud backoff allows Kubernetes nodes to backoff on HTTP 429 errors that are usually caused by exceeding Azure API limits.
 
 ## To Use
+
 Declare your kubernetes cluster API model config as you normally would, with the following requirements:
-- You must be using at minimum the `v1.6.6` version of Kubernetes to have access to the `kubernetesConfig` configuration vectors exemplified in [examples/largeclusters/kubernetes.json](https://github.com/Azure/aks-engine/blob/master/examples/largeclusters/kubernetes.json). As long as you are using a version of aks-engine `v0.3.0` or newer, your kubernetes cluster specification will fulfill this minimum version requirement.
+- You must be using at minimum the `v1.6.6` version of Kubernetes to have access to the `kubernetesConfig` configuration vectors exemplified in [examples/largeclusters/kubernetes.json](https://github.com/Azure/aks-engine/blob/master/examples/largeclusters/kubernetes.json). As long as you are using a version of AKS-Engine `v0.3.0` or newer, your kubernetes cluster specification will fulfill this minimum version requirement.
 - We recommend the use of smaller pools (e.g., count of 20) over larger pools (e.g., count of 100); produce your desired total node count with lots of pools, as opposed to as few as possible.
 - We also recommend using large vmSize configurations to reduce node counts, where appropriate. Make sure you have a defensible infrastructure justification for more nodes in terms of node count (for example as of kubernetes 1.7 there is a 100 pods per node limit), instead of opting to use more powerful nodes. Doing so reduces cluster complexity, and azure resource administrative overhead. As Kubernetes excels in binpacking pods onto available instances, vertically scaling VM sizes (more CPU/RAM) is a better approach for expanding cluster capacity, if you are not approaching the pod-per-node limit.
 
 ## Backoff configuration options
+
 The following configuration parameters are available in the `properties.orchestratorProfile.kubernetesConfig` configuration object in the api model specification:
 
 ```json