Skip to content
This repository was archived by the owner on Dec 16, 2020. It is now read-only.

Commit c65779d

Browse files
yorinasub17yorinasub17
committed
Migrate k8s-namespace and k8s-service-account modules
1 parent dcf0f0c commit c65779d

27 files changed

+1793
-0
lines changed

‎.circleci/config.yml‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
defaults: &defaults
2+
machine: true
3+
environment:
4+
GRUNTWORK_INSTALLER_VERSION: v0.0.21
5+
TERRATEST_LOG_PARSER_VERSION: v0.13.13
6+
KUBERGRUNT_VERSION: v0.1.4
7+
MODULE_CI_VERSION: v0.13.3
8+
TERRAFORM_VERSION: 0.11.8
9+
TERRAGRUNT_VERSION: NONE
10+
PACKER_VERSION: NONE
11+
GOLANG_VERSION: 1.11.2
12+
K8S_VERSION: v1.10.0 # Same as EKS
13+
KUBECONFIG: /home/circleci/.kube/config
14+
MINIKUBE_VERSION: v0.28.2 # See https://github.com/kubernetes/minikube/issues/2704
15+
MINIKUBE_WANTUPDATENOTIFICATION: "false"
16+
MINIKUBE_WANTREPORTERRORPROMPT: "false"
17+
MINIKUBE_HOME: /home/circleci
18+
CHANGE_MINIKUBE_NONE_USER: "true"
19+
20+
# Install and setup minikube
21+
# https://github.com/kubernetes/minikube#linux-continuous-integration-without-vm-support
22+
setup_minikube: &setup_minikube
23+
name: install kubectl and minikube
24+
command: |
25+
# install kubectl
26+
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/linux/amd64/kubectl
27+
chmod +x kubectl
28+
sudo mv kubectl /usr/local/bin/
29+
mkdir -p ${HOME}/.kube
30+
touch ${HOME}/.kube/config
31+
32+
# Install minikube
33+
curl -Lo minikube https://github.com/kubernetes/minikube/releases/download/${MINIKUBE_VERSION}/minikube-linux-amd64
34+
chmod +x minikube
35+
sudo mv minikube /usr/local/bin/
36+
37+
# start minikube and wait for it
38+
# Ignore warnings on minikube start command, as it will log a warning due to using deprecated localkube
39+
# bootstrapper. However, the localkube bootstrapper is necessary to run on CircleCI which doesn't have
40+
# systemd.
41+
sudo -E minikube start --vm-driver=none --kubernetes-version=${K8S_VERSION} --bootstrapper=localkube
42+
# this for loop waits until kubectl can access the api server that Minikube has created
43+
$(
44+
for i in {1..150}; do # timeout for 5 minutes
45+
kubectl get po &> /dev/null
46+
if [ $? -ne 1 ]; then
47+
break
48+
fi
49+
sleep 2
50+
done
51+
true
52+
)
53+
54+
install_gruntwork_utils: &install_gruntwork_utils
55+
name: install gruntwork utils
56+
command: |
57+
curl -Ls https://raw.githubusercontent.com/gruntwork-io/gruntwork-installer/master/bootstrap-gruntwork-installer.sh | bash /dev/stdin --version "${GRUNTWORK_INSTALLER_VERSION}"
58+
gruntwork-install --module-name "gruntwork-module-circleci-helpers" --repo "https://github.com/gruntwork-io/module-ci" --tag "${MODULE_CI_VERSION}"
59+
gruntwork-install --binary-name "terratest_log_parser" --repo "https://github.com/gruntwork-io/terratest" --tag "${TERRATEST_LOG_PARSER_VERSION}"
60+
configure-environment-for-gruntwork-module \
61+
--circle-ci-2-machine-executor \
62+
--terraform-version ${TERRAFORM_VERSION} \
63+
--terragrunt-version ${TERRAGRUNT_VERSION} \
64+
--packer-version ${PACKER_VERSION} \
65+
--use-go-dep \
66+
--go-version ${GOLANG_VERSION} \
67+
--go-src-path test \
68+
69+
70+
version: 2
71+
jobs:
72+
setup:
73+
<<: *defaults
74+
steps:
75+
- checkout
76+
- restore_cache:
77+
keys:
78+
- dep-{{ checksum "test/Gopkg.lock" }}
79+
80+
# Install gruntwork utilities
81+
- run:
82+
<<: *install_gruntwork_utils
83+
84+
- save_cache:
85+
key: dep-{{ checksum "test/Gopkg.lock" }}
86+
paths:
87+
- ./test/vendor
88+
89+
# Fail the build if the pre-commit hooks don't pass. Note: if you run pre-commit install locally, these hooks will
90+
# execute automatically every time before you commit, ensuring the build never fails at this step!
91+
- run: pip install pre-commit==1.11.2
92+
- run: pre-commit install
93+
- run: pre-commit run --all-files
94+
95+
- persist_to_workspace:
96+
root: /home/circleci
97+
paths:
98+
- project
99+
- terraform
100+
- packer
101+
102+
integration_tests:
103+
<<: *defaults
104+
steps:
105+
- attach_workspace:
106+
at: /home/circleci
107+
108+
# The weird way you have to set PATH in Circle 2.0
109+
- run: echo 'export PATH=$HOME/terraform:$HOME/packer:$PATH' >> $BASH_ENV
110+
111+
- run:
112+
<<: *install_gruntwork_utils
113+
114+
- run:
115+
<<: *setup_minikube
116+
117+
- run:
118+
name: Install kubergrunt
119+
command: gruntwork-install --binary-name "kubergrunt" --repo "https://github.com/gruntwork-io/kubergrunt" --tag "${KUBERGRUNT_VERSION}"
120+
121+
# Execute main terratests
122+
- run:
123+
name: run integration tests
124+
command: |
125+
mkdir -p /tmp/logs
126+
run-go-tests --path test --timeout 60m | tee /tmp/logs/all.log
127+
no_output_timeout: 3600s
128+
129+
- run:
130+
command: terratest_log_parser --testlog /tmp/logs/all.log --outputdir /tmp/logs
131+
when: always
132+
- store_artifacts:
133+
path: /tmp/logs
134+
- store_test_results:
135+
path: /tmp/logs
136+
137+
workflows:
138+
version: 2
139+
test-and-deploy:
140+
jobs:
141+
- setup:
142+
filters:
143+
tags:
144+
only: /^v.*/
145+
146+
- integration_tests:
147+
requires:
148+
- setup
149+
filters:
150+
tags:
151+
only: /^v.*/

‎examples/k8s-namespace-with-service-account/README.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# K8S Namespace with Service Account
2+
3+
This folder shows an example of how to create a new namespace using the [`k8s-namespace`](/modules/k8s-namespace) module, and create two new `ServiceAccounts`:
4+
5+
- One bound to `namespace-access-all` role
6+
- One bound to `namespace-access-read-only` role
7+
8+
After this example you should have a new namespace with RBAC roles that can be used to grant read-write or read-only
9+
access to the namespace, and two `ServiceAccounts` that are bound to each of the roles.
10+
11+
<!-- Maintainer's note: This example is primarily used for unit testing the underlying module -->
12+
13+
## How do you run this example?
14+
15+
To run this example, apply the Terraform templates:
16+
17+
1. Install [Terraform](https://www.terraform.io/), minimum version: `0.9.7`.
18+
1. Install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
19+
1. Open `variables.tf`, set the environment variables specified at the top of the file, and fill in any other variables
20+
that don't have a default.
21+
1. Run `terraform init`.
22+
1. Run `terraform apply`.

‎examples/k8s-namespace-with-service-account/main.tf‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2+
# CREATE A NAMESPACE WITH DEFAULT RBAC ROLES AND SERVICE ACCOUNTS BOUND TO THE ROLES
3+
# These templates show an example of how to create a Kubernetes namespace with a set of default RBAC roles, and
4+
# ServiceAccounts that are bound to each default role.
5+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6+
7+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8+
# CONFIGURE OUR KUBERNETES CONNECTIONS
9+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10+
11+
provider "kubernetes" {
12+
config_context = "${var.kubectl_config_context_name}"
13+
config_path = "${var.kubectl_config_path}"
14+
}
15+
16+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
17+
# CREATE THE NAMESPACE WITH RBAC ROLES
18+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
19+
20+
module "namespace" {
21+
# When using these modules in your own templates, you will need to use a Git URL with a ref attribute that pins you
22+
# to a specific version of the modules, such as the following example:
23+
# source = "git::git@github.com:gruntwork-io/package-k8s.git//modules/k8s-namespace?ref=v0.1.0"
24+
source = "../../modules/k8s-namespace"
25+
26+
kubectl_config_context_name = "${var.kubectl_config_context_name}"
27+
kubectl_config_path = "${var.kubectl_config_path}"
28+
name = "${var.name}"
29+
}
30+
31+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
32+
# CREATE THE SERVICE ACCOUNTS
33+
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
34+
35+
module "service_account_access_all" {
36+
# When using these modules in your own templates, you will need to use a Git URL with a ref attribute that pins you
37+
# to a specific version of the modules, such as the following example:
38+
# source = "git::git@github.com:gruntwork-io/package-k8s.git//modules/k8s-service-account?ref=v0.1.0"
39+
source = "../../modules/k8s-service-account"
40+
41+
kubectl_config_context_name = "${var.kubectl_config_context_name}"
42+
kubectl_config_path = "${var.kubectl_config_path}"
43+
name = "${var.name}-admin"
44+
namespace = "${module.namespace.name}"
45+
rbac_roles = ["${module.namespace.rbac_access_all_role}"]
46+
47+
# How to tag the service account with a label
48+
labels = {
49+
role = "admin"
50+
}
51+
}
52+
53+
module "service_account_access_read_only" {
54+
# When using these modules in your own templates, you will need to use a Git URL with a ref attribute that pins you
55+
# to a specific version of the modules, such as the following example:
56+
# source = "git::git@github.com:gruntwork-io/package-k8s.git//modules/k8s-service-account?ref=v0.1.0"
57+
source = "../../modules/k8s-service-account"
58+
59+
kubectl_config_context_name = "${var.kubectl_config_context_name}"
60+
kubectl_config_path = "${var.kubectl_config_path}"
61+
name = "${var.name}-read-only"
62+
namespace = "${module.namespace.name}"
63+
rbac_roles = ["${module.namespace.rbac_access_read_only_role}"]
64+
65+
# How to tag the service account with a label
66+
labels = {
67+
role = "monitor"
68+
}
69+
}

‎examples/k8s-namespace-with-service-account/outputs.tf‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
output "name" {
2+
description = "Name of the created namespace"
3+
value = "${module.namespace.name}"
4+
}
5+
6+
output "rbac_access_all_role" {
7+
description = "The name of the RBAC role that grants admin level permissions on the namespace."
8+
value = "${module.namespace.rbac_access_all_role}"
9+
}
10+
11+
output "rbac_access_read_only_role" {
12+
description = "The name of the RBAC role that grants read only permissions on the namespace."
13+
value = "${module.namespace.rbac_access_read_only_role}"
14+
}
15+
16+
output "service_account_access_all" {
17+
description = "The name of the ServiceAccount that has admin level permissions."
18+
value = "${module.service_account_access_all.name}"
19+
}
20+
21+
output "service_account_access_read_only" {
22+
description = "The name of the ServiceAccount that has read only level permissions."
23+
value = "${module.service_account_access_read_only.name}"
24+
}

‎examples/k8s-namespace-with-service-account/variables.tf‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# ---------------------------------------------------------------------------------------------------------------------
2+
# MODULE PARAMETERS
3+
# These variables are expected to be passed in by the operator
4+
# ---------------------------------------------------------------------------------------------------------------------
5+
6+
variable "name" {
7+
description = "Name of the namespace to be created"
8+
}
9+
10+
variable "kubectl_config_context_name" {
11+
description = "The config context to use when authenticating to the Kubernetes cluster. If empty, defaults to the current context specified in the kubeconfig file."
12+
default = ""
13+
}
14+
15+
variable "kubectl_config_path" {
16+
description = "The path to the config file to use for kubectl. If empty, defaults to $HOME/.kube/config"
17+
default = "~/.kube/config"
18+
}

‎modules/k8s-namespace/README.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# K8S Namespace Module
2+
3+
This Terraform Module manages Kubernetes
4+
[`Namespaces`](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/). In addition to creating
5+
namespaces, this module will create a set of default RBAC roles restricted to that namespace. The following roles will
6+
be provided by this module:
7+
8+
- `namespace-access-all`: Admin level permissions in the namespace. Ability to read, write, and delete all resources in
9+
the namespace.
10+
- `namespace-access-read-only`: Read only permissions to all resources in the namespace.
11+
12+
13+
## How do you use this module?
14+
15+
* See the [root README](/README.md) for instructions on using Terraform modules.
16+
* This module uses [the `kubernetes` provider](https://www.terraform.io/docs/providers/kubernetes/index.html).
17+
* See the [examples](/examples) folder for example usage.
18+
* See [variables.tf](./variables.tf) for all the variables you can set on this module.
19+
* See [outputs.tf](./outputs.tf) for all the variables that are outputed by this module.
20+
21+
22+
## What is a Namespace?
23+
24+
A `Namespace` is a Kubernetes resource that can be used to create a virtual environment in your cluster to separate
25+
resources. It allows you to scope resources in your cluster to provide finer grained permission control and resource
26+
quotas.
27+
28+
For example, suppose that you have two different teams managing separate services independently, such that the team
29+
should not be allowed to update or modify the other teams' services. In such a scenario, you would use namespaces to
30+
separate the resources between each team, and implement RBAC roles that only grant access to the namespace if you reside
31+
in the team that manages it.
32+
33+
To illustrate this, let's assume that we have a team that manages
34+
the application services related to the core product (named `core`) and another team managing analytics services (named
35+
`analytics`). Let's also assume that we have already created RBAC groups for each team, named `core-group` and
36+
`analytics-group`.
37+
38+
We create the namespaces using this module:
39+
40+
```
41+
module "core_namespace" {
42+
source = "git::git@github.com:gruntwork-io/package-k8s.git//modules/k8s-namespace?ref=v0.1.0"
43+
name = "core"
44+
}
45+
46+
module "analytics_namespace" {
47+
source = "git::git@github.com:gruntwork-io/package-k8s.git//modules/k8s-namespace?ref=v0.1.0"
48+
name = "analytics"
49+
}
50+
```
51+
52+
In addition to creating namespaces, this will also create a set of RBAC roles that can then be bound to user and group
53+
entities to explicitly grant permissions to access that namespace.
54+
55+
We can then use `kubectl` to bind the roles to the groups:
56+
```
57+
---
58+
kind: RoleBinding
59+
apiVersion: rbac.authorization.k8s.io/v1
60+
metadata:
61+
name: core-role-binding
62+
namespace: core
63+
subjects:
64+
- kind: Group
65+
name: core
66+
apiGroup: rbac.authorization.k8s.io
67+
roleRef:
68+
kind: Role
69+
name: core-access-all
70+
apiGroup: rbac.authorization.k8s.io
71+
---
72+
kind: RoleBinding
73+
apiVersion: rbac.authorization.k8s.io/v1
74+
metadata:
75+
name: analytics-role-binding
76+
namespace: analytics
77+
subjects:
78+
- kind: Group
79+
name: analytics
80+
apiGroup: rbac.authorization.k8s.io
81+
roleRef:
82+
kind: Role
83+
name: analytics-access-all
84+
apiGroup: rbac.authorization.k8s.io
85+
```
86+
87+
When we apply this config with `kubectl`, users that are associated with the `core` RBAC group can now create and access
88+
resources deployed in the `core` namespace, and the `analytics` group can access resources in the `analytics` namespace.
89+
However, members of the `core` team can not access resources in the `analytics` namespace and vice versa.
90+
91+
To summarize, use namespaces to:
92+
93+
- Implement finer grained access control over deployed resources.
94+
- Implement [resource quotas](https://kubernetes.io/docs/concepts/policy/resource-quotas/) to restrict how much of the
95+
cluster can be utilized by each team.
96+
97+
98+
## Why is this a Terraform Module and not a Helm Chart?
99+
100+
This module uses Terraform to manage the `Namespace` and RBAC role resources instead of using Helm to support the use case of
101+
setting up Helm. When setting up the Helm server, you will want to setup a `Namespace` and `ServiceAccount` for the Helm
102+
server to be deployed with. This leads to a chicken and egg problem, where the `Namespace` and `ServiceAccount` needs to
103+
be created before Helm is available for use. As such, we rely on Terraform to set these core resources up.

0 commit comments

Comments
 (0)