Skip to content

Commit 6f3a66b

Browse files
timtorChentimtorChen
committed
refactor: change the controlplane node to intel nuc
- doc: update readme - fix: cephcluster wrong cluster addresses - chore: config ceph internal network in cephcluster CR and remove rook-config-override - chore: change kubeprism port to default 7445
1 parent 385450f commit 6f3a66b

File tree

16 files changed

+147
-118
lines changed

16 files changed

+147
-118
lines changed

‎README.md‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ This is a mono repository for my home infrastructure and Kubernetes cluster. I u
4040

4141
| | Name | Description |
4242
| :--------------------------------------------------------------------------------------------------------------------------------------------------: | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
43-
| <img width="32" src="https://www.talos.dev/images/logo.svg"> | [Talos](https://github.com/siderolabs/talos) | Immutable Linux distribution for Kubernetes. |
43+
| <img width="32" src="https://avatars.githubusercontent.com/u/13804887?v=4"> | [Talos](https://github.com/siderolabs/talos) | Immutable Linux distribution for Kubernetes. |
4444
| <img width="28" src="https://raw.githubusercontent.com/cncf/artwork/main/projects/flux/icon/color/flux-icon-color.svg"> | [Flux](https://github.com/fluxcd/flux2) | Gitops tool to reconcile sources from Git repository to Kubernetes. |
4545
| <img width="32" src="https://raw.githubusercontent.com/cncf/artwork/main/projects/kyverno/icon/color/kyverno-icon-color.svg"> | [Kyverno](https://github.com/kyverno/kyverno) | Kubernetes policy manager. |
4646
| <img width="32" src="https://raw.githubusercontent.com/cncf/artwork/main/projects/cilium/icon/color/cilium_icon-color.svg"> | [Cilium](https://github.com/cilium/cilium) | Advanced networking. |
@@ -61,12 +61,12 @@ This is a mono repository for my home infrastructure and Kubernetes cluster. I u
6161
| :------------ | :----------------------------------------- | :-------------- |
6262
| Github | Code repository and automation chores/jobs | Free |
6363
| JumpCloud | SSO identity provider | Free |
64-
| Cloudflare | Domain registrar and tunnel | $10/year |
65-
| Backblaze | S3 bucket for buckup | ~$1/month |
64+
| Cloudflare | Domain registrar and tunnel | $12/year |
65+
| Backblaze | S3 bucket for buckup | ~$5/month |
6666
| AWS | Parameter storage and terraform backend | Free |
6767
| Grafana Cloud | External montoring | Free |
6868
| Let's Encrypt | Public certificate authroity | Free |
69-
| | | Total ~$22/year |
69+
| | | Total ~$6/month |
7070

7171
## 🔧 Hardware
7272

@@ -75,14 +75,14 @@ This is a mono repository for my home infrastructure and Kubernetes cluster. I u
7575
<img src="docs/src/rack-20241103.jpg" width="400px"/>
7676
</details>
7777

78-
| Device | Description | Count | RAM | Disk |
79-
| ------------------------------- | ---------------------------- | ----- | -------------------------- | ------------------------------------------------------------------------- |
80-
| Askey RTF8207W | Chunghwa Telecom modem | 1 | | |
81-
| Mikrotik RB4011iGS+RM | Router | 1 | | |
82-
| Mikrotik CRS328-24P-4S+RM | PoE Switch | 1 | | |
83-
| Raspberry Pi 4B with PoE hat | Kubernetes control planes | 1 | 8GB | 960GB SSD Micron 5200 |
84-
| Intel NUC11TNHi50L | Kubernetes worker nodes | 3 | 16\*2 Mircon CT16G4SFRA32A | <div>• OS: 960GB SSD Mircon 5300<div/><div>• Data: 960GB NVMe Mircon 7450 |
85-
| Ubiquiti Power Distribution Pro | 16p Switched and metered PDU | 1 | | |
78+
| Device | Description | Count | RAM | Disk |
79+
| ------------------------------- | ---------------------------- | ----- | ---------------------------- | ------------------------------------------------------------------------- |
80+
| Askey RTF8207W | Chunghwa Telecom modem | 1 | | |
81+
| Mikrotik RB4011iGS+RM | Router | 1 | | |
82+
| Mikrotik CRS328-24P-4S+RM | PoE Switch | 1 | | |
83+
| Intel NUC12WSHi5 | Kubernetes control planes | 1 | 16GB Mircon CT16G4SFRA32A | 960GB SSD Micron 5200 |
84+
| Intel NUC11TNHi50L | Kubernetes worker nodes | 3 | 16GB\*2 Mircon CT16G4SFRA32A | <div>• OS: 960GB SSD Mircon 5300<div/><div>• Data: 960GB NVMe Mircon 7450 |
85+
| Ubiquiti Power Distribution Pro | 16p Switched and metered PDU | 1 | | |
8686

8787
## 🤝 Acknowledgments
8888

‎kubernetes/kube-system/cilium/release.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ spec:
4747
enabled: false
4848
hostRoot: /sys/fs/cgroup
4949
k8sServiceHost: localhost
50-
k8sServicePort: 7745
50+
k8sServicePort: 7445
5151
hubble:
5252
enabled: true
5353
relay:

‎kubernetes/rook-ceph/cluster/cephcluster.yaml‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,17 @@ spec:
99
cephVersion:
1010
image: quay.io/ceph/ceph:v19.2.2
1111
dataDirHostPath: /var/lib/rook
12+
network:
13+
provider: host
14+
addressRanges:
15+
public:
16+
- 192.168.253.11/21
17+
- 192.168.253.12/21
18+
- 192.168.253.13/21
19+
cluster:
20+
- 192.168.253.101/32
21+
- 192.168.253.102/32
22+
- 192.168.253.103/32
1223
mon:
1324
count: 3
1425
allowMultiplePerNode: false
@@ -58,8 +69,6 @@ spec:
5869
memory: 64Mi
5970
requests:
6071
memory: 64Mi
61-
network:
62-
provider: host
6372
dashboard:
6473
enabled: true
6574
monitoring:

‎kubernetes/rook-ceph/cluster/rook-config-override.yaml‎

Lines changed: 0 additions & 16 deletions
This file was deleted.

‎kubernetes/rook-ceph/kustomization.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@ resources:
88
- app/release.yaml
99
# cluster
1010
- cluster/cephcluster.yaml
11-
- cluster/rook-config-override.yaml
1211
# block
1312
- storage/block/cephblock.yaml
1413
- storage/block/snapshotclass.yaml

‎talos/.taskfile.yaml‎

Lines changed: 24 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,8 @@ silent: true
55

66
tasks:
77
# schematics id
8-
get-nuc11tnhi50l-schematic-id:
9-
cmd: curl -s -X POST --data-binary @schematics/nuc11tnhi50l.yaml https://factory.talos.dev/schematics | jq --raw-output .id
10-
get-raspi-schematic-id:
11-
cmd: curl -s -X POST --data-binary @schematics/raspi.yaml https://factory.talos.dev/schematics | jq --raw-output .id
8+
get-intelnuc-schematic-id:
9+
cmd: curl -s -X POST --data-binary @schematics/intelnuc.yaml https://factory.talos.dev/schematics | jq --raw-output .id
1210

1311
# talos task functions
1412
render:
@@ -34,7 +32,7 @@ tasks:
3432
IP="{{.IP}}"
3533
3634
echo "> (dry-run) Apply configuration on $IP $TALOS_ROLE/$TALOS_NODE"
37-
talosctl apply-config --dry-run -n "$IP" -f <(echo "$CONFIG")
35+
talosctl apply-config --dry-run -n "$IP" -f <(echo "$CONFIG") {{.CLI_ARGS}}
3836
3937
apply:
4038
internal: true
@@ -48,7 +46,7 @@ tasks:
4846
IP="{{.IP}}"
4947
5048
echo "> Apply configuration on $IP $TALOS_ROLE/$TALOS_NODE"
51-
talosctl apply-config -n "$IP" -f <(echo "$CONFIG")
49+
talosctl apply-config -n "$IP" -f <(echo "$CONFIG") {{.CLI_ARGS}}
5250
5351
upgrade:
5452
internal: true
@@ -71,7 +69,7 @@ tasks:
7169
prompt: Shutdown the node {{.TALOS_ROLE}}/{{.TALOS_NODE}}... continue?
7270
cmd: |
7371
IP="{{.IP}}"
74-
talosctl shutdonw -n "$IP"
72+
talosctl shutdown -n "$IP"
7573
7674
reset:
7775
internal: true
@@ -84,34 +82,39 @@ tasks:
8482
--system-labels-to-wipe=META \
8583
--reboot --graceful -n "$IP"
8684
87-
# pi4b-1
88-
render-pi4b-1:
85+
# bootstrap
86+
bootstrap:
87+
cmd: |
88+
talosctl bootstrap {{.CLI_ARGS}}
89+
90+
# nuc12wshi5-1
91+
render-nuc12wshi5-1:
8992
cmds:
9093
- task: render
91-
vars: &pi4b-1-vars
94+
vars: &nuc12wshi5-1-vars
9295
IP: 192.168.253.1
93-
TALOS_NODE: pi4b-1
96+
TALOS_NODE: nuc12wshi5-1
9497
TALOS_ROLE: controlplane
95-
plan-pi4b-1:
98+
plan-nuc12wshi5-1:
9699
cmds:
97100
- task: plan
98-
vars: *pi4b-1-vars
99-
apply-pi4b-1:
101+
vars: *nuc12wshi5-1-vars
102+
apply-nuc12wshi5-1:
100103
cmds:
101104
- task: apply
102-
vars: *pi4b-1-vars
103-
upgrade-pi4b-1:
105+
vars: *nuc12wshi5-1-vars
106+
upgrade-nuc12wshi5-1:
104107
cmds:
105108
- task: upgrade
106-
vars: *pi4b-1-vars
107-
shutdown-pi4b-1:
109+
vars: *nuc12wshi5-1-vars
110+
shutdown-nuc12wshi5-1:
108111
cmds:
109112
- task: shutdown
110-
vars: *pi4b-1-vars
111-
reset-pi4b-1:
113+
vars: *nuc12wshi5-1-vars
114+
reset-nuc12wshi5-1:
112115
cmds:
113116
- task: reset
114-
vars: *pi4b-1-vars
117+
vars: *nuc12wshi5-1-vars
115118

116119
# nuc11tnhi50l-1
117120
render-nuc11tnhi50l-1:

‎talos/schematics/nuc11tnhi50l.yaml‎ renamed to ‎talos/schematics/intelnuc.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@ overlay: {}
33
customization:
44
systemExtensions:
55
officialExtensions:
6-
- siderolabs/i915-ucode
6+
- siderolabs/i915

‎talos/schematics/raspi.yaml‎

Lines changed: 0 additions & 7 deletions
This file was deleted.

‎talos/templates/nodes/nuc11tnhi50l-1.tmpl‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,24 @@ machine:
33
type: worker
44
network:
55
hostname: nuc11tnhi50l-1
6+
interfaces:
7+
# external network
8+
- interface: eth0
9+
addresses:
10+
- 192.168.253.11/21
11+
routes:
12+
- network: 0.0.0.0/0
13+
gateway: 192.168.248.1
14+
# internal ceph communication
15+
- interface: eth1
16+
addresses:
17+
- 192.168.253.101/21
18+
routes:
19+
- network: 192.168.253.101/32
20+
- network: 192.168.253.102/32
21+
- network: 192.168.253.103/32
22+
nameservers:
23+
- 192.168.248.1
624
udev:
725
# check /usr/etc/udev/rules.d/99-talos.rules
826
rules:
@@ -11,7 +29,7 @@ machine:
1129
install:
1230
disk: /dev/sda
1331
# renovate: depName=ghcr.io/siderolabs/installer
14-
image: factory.talos.dev/installer/ed036d0640097a4e7af413ee089851a12963cd2e2e1715f8866d551d17c2ec62:v1.9.0
32+
image: factory.talos.dev/installer/dc8730aa8cc7bfa5ef7e2b3284248f2631135b2faf4ae11aa997a0c1987b0eee:v1.9.0
1533
kubelet:
1634
image: ghcr.io/siderolabs/kubelet:v1.32.0
1735
{{ end }}

‎talos/templates/nodes/nuc11tnhi50l-2.tmpl‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,24 @@ machine:
33
type: worker
44
network:
55
hostname: nuc11tnhi50l-2
6+
interfaces:
7+
# external network
8+
- interface: eth0
9+
addresses:
10+
- 192.168.253.12/21
11+
routes:
12+
- network: 0.0.0.0/0
13+
gateway: 192.168.248.1
14+
# internal ceph communication
15+
- interface: eth1
16+
addresses:
17+
- 192.168.253.102/21
18+
routes:
19+
- network: 192.168.253.101/32
20+
- network: 192.168.253.102/32
21+
- network: 192.168.253.103/32
22+
nameservers:
23+
- 192.168.248.1
624
udev:
725
# check /usr/etc/udev/rules.d/99-talos.rules
826
rules:
@@ -11,7 +29,7 @@ machine:
1129
install:
1230
disk: /dev/sda
1331
# renovate: depName=ghcr.io/siderolabs/installer
14-
image: factory.talos.dev/installer/ed036d0640097a4e7af413ee089851a12963cd2e2e1715f8866d551d17c2ec62:v1.9.0
32+
image: factory.talos.dev/installer/dc8730aa8cc7bfa5ef7e2b3284248f2631135b2faf4ae11aa997a0c1987b0eee:v1.9.0
1533
kubelet:
1634
image: ghcr.io/siderolabs/kubelet:v1.32.0
1735
{{ end }}

0 commit comments

Comments
 (0)