Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to MinIO 2021-10-27 #1

Merged
merged 2 commits into from
Nov 7, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions 200-secrets.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ metadata:
labels:
app: minio
data:
MINIO_ACCESS_KEY: bXktYWNjZXNzLWtleQ==
MINIO_SECRET_KEY: bXlYWFh4eHgvc2VjcmV0WFhYWHh4eC9rZXlYWHh4eA==
MINIO_ROOT_USER: bXktYWNjZXNzLWtleQ==
MINIO_ROOT_PASSWORD: bXlYWFh4eHgvc2VjcmV0WFhYWHh4eC9rZXlYWHh4eA==
3 changes: 3 additions & 0 deletions 300-service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,8 @@ spec:
- protocol: TCP
name: minio
port: 9000
- protocol: TCP
name: minio-console
port: 9001
selector:
app: minio
18 changes: 12 additions & 6 deletions 400-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,29 +28,35 @@ spec:
- hdd
containers:
- name: minio
image: minio/minio:RELEASE.2020-10-28T08-16-50Z-arm64
args: ['server', '/data']
image: minio/minio:RELEASE.2021-10-27T16-29-42Z
args: ['server', '/data', '--console-address', ':9001']
ports:
- name: minio
containerPort: 9000
volumeMounts:
- name: s3-pv-storage
mountPath: /data
env:
- name: MINIO_SERVER_URL
value: https://minio.example.com
- name: MINIO_BROWSER_REDIRECT_URL
value: https://console.minio.example.com
- name: MINIO_IDENTITY_OPENID_CONFIG_URL
value: https://keycloak.example.com/auth/realms/home/.well-known/openid-configuration
- name: MINIO_IDENTITY_OPENID_CLIENT_ID
value: minio
- name: MINIO_ACCESS_KEY
- name: MINIO_IDENTITY_OPENID_CLIENT_SECRET
value: cad5f999-81e6-4791-afef-86ddd18ae3df
- name: MINIO_ROOT_USER
valueFrom:
secretKeyRef:
name: minio
key: MINIO_ACCESS_KEY
- name: MINIO_SECRET_KEY
key: MINIO_ROOT_USER
- name: MINIO_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: minio
key: MINIO_SECRET_KEY
key: MINIO_ROOT_PASSWORD
volumes:
- name: s3-pv-storage
persistentVolumeClaim:
Expand Down
7 changes: 6 additions & 1 deletion 500-ingressroute.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,10 @@ spec:
services:
- name: minio
port: 9000
- match: Host(`console.minio.example.com`)
kind: Rule
services:
- name: minio
port: 9001
tls:
certResolver: godaddy
certResolver: tlsresolver
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2021 Simon Leigh

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
66 changes: 46 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,28 @@ My uses of this include:
This repository contains instructions and Kubernetes manifest files for
deploying MinIO into a K3s cluster.

**UPDATE: 7 Nov 2021**: This repository has been upgraded to
RELEASE.2021-10-27T16-29-42Z from MinIO RELEASE.2020-10-28T08-16-50Z-arm64.
There have been a number of changes between versions so check the release notes
for more information if you are upgrading. The configuration and instructions
from this repository for the previous release are still available from the
[v1.0.0 tag].

## K3s Deployment

[K3s] is a lightweight, certified Kubernetes distribution, for production
workloads from Rancher Labs. I run K3s on my Raspberry Pi cluster, into which
MinIO has been deployed. See [raspberry-pi-k3s-homelab] for more information on
this.

### Create Access Key and Secret Key
### Create Root User and Password

The access key and secret key are used as the username and password when
accessing MinIO. I have used a similar format as the AWS secret key and secret
access key, which also highlighted a couple of gotchas with the S3 url format
when integrating some apps with this.
The username and password of the root user need to be created as secrets and
provided as environment variables. I have used a similar format as the AWS
secret key and secret access key, as this was the pattern used for earlier MinIO
naming conventions. This highlighted a couple of gotchas with the S3 url format
when integrating some apps with this, which I have documented in this README.md
file.

These need to be in base64 format as they will be added to the
`200-secrets.yaml` file to be created as secrets in Kubernetes.
Expand Down Expand Up @@ -62,6 +71,27 @@ affinity:
- hdd
```

### Ingress Route

My K3s cluster uses [Traefik v2] as the Kubernetes ingress controller and the
`IngressRoute` resource it provides. You can just as easily use the standard
Kubernetes `Ingress` resource, or other such ingress controller annotations or
configuration methods.

The `500-ingressroute.yaml` file specifies the usage of TLS and associated
certificate resolver. You should always use HTTPS when accessing resources,
especially over the internet. You can update that file as necessary to work with
your deployment, e.g. using a different Traefik endpoint and removing the `tls`
section. In this example `tlsresolver` is a certificate resolver configured in
Traefik to use Let's Encrypt and the `tlsChallenge` type.

Note that there is a `console.minio.example.com` entry as well as a
`minio.example.com`. When accessing MinIO via the browser using
<https://minio.example.com> you will be automatically redirected to the MinIO
Console at <https://console.minio.example.com>. The MinIO Console is used for
both administering the system, as well as standard user access when navigating
buckets.

### OpenID Connect and Keycloak

MinIO supports authentication using OpenID Connect and providers such as
Expand All @@ -74,6 +104,8 @@ variables can be removed from the `400-deployment.yaml` file.
value: https://keycloak.example.com/auth/realms/home/.well-known/openid-configuration
- name: MINIO_IDENTITY_OPENID_CLIENT_ID
value: minio
- name: MINIO_IDENTITY_OPENID_CLIENT_SECRET
value: cad5f999-81e6-4791-afef-86ddd18ae3df
```

If integrating with Keycloak the MinIO docs on this were pretty good, see
Expand All @@ -84,19 +116,6 @@ Keycloak.
See the section further down in the README file for information on setting up a
policy for authorization to MinIO resources using JWT with Keycloak.

### Ingress Route

My K3s cluster uses [Traefik v2] as the Kubernetes ingress controller and the
`IngressRoute` resource it provides. You can just as easily use the standard
Kubernetes `Ingress` resource, or other such ingress controller annotations or
configuration methods.

The `500-ingressroute.yaml` file specifies the usage of TLS and associated
certificates. You should always use HTTPS when accessing resources, especially
over the internet. You can update that file as necessary to work with your
deployment, e.g. using a different Traefik endpoint and removing the `tls`
section.

## MinIO Client

The [MinIO Client (mc)] can be easily used as to access and administer your
Expand Down Expand Up @@ -229,7 +248,7 @@ JWT that was provided during the OpenID Connect login. This is similar to the
previous `readwriteusers` policy except that the user principal is identified by
the `preferred_username` claim within the JWT.

This policy is **not** set on the `users` group. MinIO will select and enforece
This policy is **not** set on the `users` group. MinIO will select and enforce
this policy based on the `readwriteusersjwt` name specified in the `policy`
claim of the token.

Expand Down Expand Up @@ -294,7 +313,7 @@ authenticated it will retrieve "readwriteusersjwt" from the user's

The below error in the Minio logs may be seen when failing to authenticate using
JWT. This can mean that the `policy` claim is missing. Check to ensure that the
`policy` Claim Mapping has been added to the cient in Keycloak. Check that this
`policy` Claim Mapping has been added to the client in Keycloak. Check that this
is mapped to a user attribute belonging to the user attempting to authenticate,
and that the value of this attribute on the user matches the Minio policy name
for the JWT policy, e.g. `readwriteusersjwt`.
Expand Down Expand Up @@ -349,6 +368,10 @@ aws:

<https://github.com/grafana/loki/issues/1434>

## License

[![MIT license]](https://lbesson.mit-license.org/)

[docker registry]: https://docs.docker.com/registry/
[external hard drive for persistent storage]:
https://github.com/sleighzy/raspberry-pi-k3s-homelab/blob/main/k3s.md#external-hard-drive-for-persistent-storage
Expand All @@ -358,7 +381,10 @@ aws:
[local path provisioner]: https://rancher.com/docs/k3s/latest/en/storage/
[minio]: https://min.io/
[minio client (mc)]: https://docs.min.io/docs/minio-client-complete-guide.html
[mit license]: https://img.shields.io/badge/License-MIT-blue.svg
[raspberry-pi-k3s-homelab]:
https://github.com/sleighzy/raspberry-pi-k3s-homelab/blob/main/k3s.md
[restic]: https://restic.net/
[traefik v2]: https://traefik.io/traefik/
[v1.0.0 tag]:
https://github.com/sleighzy/k3s-minio-deployment/releases/tag/v1.0.0