Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion admin/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/192d-Wing/WinDep/admin

go 1.25.11
go 1.26.5

require (
github.com/gofiber/fiber/v2 v2.52.13
Expand Down
2 changes: 1 addition & 1 deletion api/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/192d-Wing/WinDep/api

go 1.25.11
go 1.26.5

require (
github.com/ansrivas/fiberprometheus/v2 v2.17.0
Expand Down
2 changes: 1 addition & 1 deletion image/Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# so CGO stays OFF, the binary stays static, and it cross-compiles for every arch
# without OpenSSL/CGO/QEMU. GOFIPS140 also bakes the runtime fips140 GODEBUG to "on".

ARG BUILDER_IMAGE=docker.io/library/golang:1.25-alpine
ARG BUILDER_IMAGE=docker.io/library/golang:1.26.5-alpine
# Runtime base — set per architecture by the build matrix:
# amd64 -> registry1.dso.mil/ironbank/redhat/ubi/ubi10:10.2
# arm64 -> registry1.dso.mil/ironbank/redhat/ubi/ubi10:10.2-arm64
Expand Down
2 changes: 1 addition & 1 deletion image/Containerfile.admin
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# -t ghcr.io/192d-wing/windep-admin:0.1.1-amd64 admin

ARG NODE_IMAGE=docker.io/library/node:20-alpine
ARG BUILDER_IMAGE=docker.io/library/golang:1.25-alpine
ARG BUILDER_IMAGE=docker.io/library/golang:1.26.5-alpine
ARG BASE_IMAGE=registry1.dso.mil/ironbank/redhat/ubi/ubi10:10.2
ARG GOFIPS140=v1.0.0

Expand Down
141 changes: 95 additions & 46 deletions platform/README.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,92 @@
# WinDep platform manifests (Kustomize)
# WinDep platform (Helm chart)

Deploys the [telemetry API](../api/) with a Cilium **anycast** VIP, on either full
Kubernetes or **k3s**.
Deploys the full WinDep control plane — the [telemetry/ZTP API](../api/) (`windep-api`),
the nginx **deploy server** that serves the boot/WIM/config payload tree (`windep-web`), and
the Cloudscape **admin** console (`windep-admin`) — behind Cilium **anycast** VIPs, on either
full Kubernetes or **k3s**.

```
platform/
├─ base/ # environment-agnostic manifests
├─ k8s/ # Deployment, Service, HPA, PDB, Namespace
│ └─ cilium/ # LB IP pool, BGP peering (L2 announcement alt)
├─ components/
└─ anycast/ # shared replacements: vars.yaml -> manifests
└─ overlays/
├─ example/ # full Kubernetes cluster
└─ k3s/ # k3s edge cluster
platform/windep/
├─ Chart.yaml
├─ values.yaml # documented defaults
├─ values-k3s.yaml # k3s edge site
├─ values-example.yaml # full Kubernetes site
└─ templates/
├─ api/ web/ admin/ # the three workloads (+ Service/PVC/PDB/HPA/NetworkPolicy)
├─ cilium/ # LB IP pool + BGP advertisement (L2 announcement alternative)
└─ tls/ # cert-manager ACME Issuer + Certificates
```

Each overlay supplies a single **`vars.yaml`** (the variable file) with that site's anycast
addresses and its BGP **advertise group**; the `anycast` component injects them via Kustomize
`replacements`. BGP **peering** (ASNs, neighbors) is **cluster-owned** — a cluster-wide
Per-site specifics (VIPs, StorageClasses, admin allow-list, image tags, ACME server) live in a
`values-<site>.yaml`. BGP **peering** (ASNs, neighbors) is **cluster-owned** — a cluster-wide
`CiliumBGPClusterConfig` peers upstream and its `CiliumBGPPeerConfig`s export any
`CiliumBGPAdvertisement` labeled `advertise: <group>`. WinDep therefore ships only a
`CiliumBGPAdvertisement` (labeled with that group), not a peering policy.
`CiliumBGPAdvertisement` labeled `advertise: <group>`. The chart therefore ships only a
`CiliumBGPAdvertisement` (labeled with `networking.advertiseGroup`), never a peering policy.

```bash
kubectl kustomize platform/overlays/example # preview
kubectl apply -k platform/overlays/example # apply (full k8s)
kubectl apply -k platform/overlays/k3s # apply (k3s)
helm template windep platform/windep -f platform/windep/values-k3s.yaml # preview
helm install windep platform/windep -n windep --create-namespace \
-f platform/windep/values-k3s.yaml # apply (k3s)
helm install windep platform/windep -n windep --create-namespace \
-f platform/windep/values-example.yaml # apply (full k8s)
helm upgrade windep platform/windep -n windep -f platform/windep/values-k3s.yaml # roll an update
```

Copy an overlay per environment and edit its `vars.yaml` + the image tag in `kustomization.yaml`.
Copy a values file per environment and edit its VIPs, StorageClass, `admin.allowedCIDRs`,
`images.*.tag`, and the `certManager.issuer.acme` block.

---

## Full Kubernetes
## Prerequisites

Prereqs: Cilium as CNI with `bgpControlPlane.enabled=true`, and metrics-server (for the HPA).
- **Cilium** as CNI with `bgpControlPlane.enabled=true` (or `l2announcements.enabled=true` for
`networking.mode: l2`), providing the `CiliumLoadBalancerIPPool` / `CiliumBGPAdvertisement` /
`CiliumL2AnnouncementPolicy` CRDs.
- **metrics-server** (for the API HPA).
- **cert-manager** installed cluster-wide when `certManager.enabled: true` (the default).
- **RWX StorageClass** for the payload PV (`storage.webRoot`) — NFS, Longhorn share-manager,
CephFS. `local-path` (k3s default) is RWO and will not bind it.

### Secrets (not created by the chart)

```bash
kubectl create namespace windep
kubectl -n windep create secret tls windep-api-tls --cert=api.crt --key=api.key
kubectl apply -k platform/overlays/example
# Iron Bank pull secret for the windep-web nginx image (Repo One creds):
kubectl -n windep create secret docker-registry repo1-pull \
--docker-server=registry1.dso.mil \
--docker-username=<REPO1_USER> --docker-password=<REPO1_TOKEN>

# Optional app secrets. Set CONFIG_KEY once WinPE carries the matching key:
kubectl -n windep create secret generic windep-admin-config-key --from-literal=key="$(openssl rand -base64 32)"
kubectl -n windep create secret generic windep-admin-auth --from-literal=token="<bearer>"
```

TLS is handled by cert-manager (below). To manage TLS yourself instead, set
`certManager.enabled: false` and pre-create `windep-{api,web,admin}-tls`
(`kubectl create secret tls ... --cert=... --key=...`).

---

## TLS via cert-manager ACME

With `certManager.enabled: true` the chart renders an ACME `Issuer` (or `ClusterIssuer`) and a
`Certificate` per service, writing into the `windep-{api,web,admin}-tls` secrets the workloads
mount. cert-manager issues and auto-renews them — no manual cert rotation.

> **The ACME server MUST be your internal ACME CA** (e.g. step-ca / smallstep), so issued certs
> chain to the internal root **baked into `boot.wim`**. WinPE trusts only that root — a public
> Let's Encrypt cert would fail validation on every WinPE HTTPS fetch. cert-manager supports the
> **DNS-01** and **HTTP-01** solvers only (not TLS-ALPN-01). Set `certManager.issuer.acme.solvers`
> accordingly — `platform/windep/values.yaml` documents both a DNS-01 (`rfc2136`) stub and an
> HTTP-01 (ingress) example. DNS-01 needs no inbound reachability (good for isolated VLANs);
> HTTP-01 is simpler but requires an Ingress controller (this chart ships none) and the internal
> ACME server reaching the deploy VIP on :80.

Prereqs the ACME flow needs: the ACME account private-key secret is created by cert-manager on
first use; a DNS-01 solver typically needs a credential secret (e.g. `windep-acme-tsig` for
rfc2136) — create it per your provider.

Check issuance: `kubectl -n windep get issuer,certificate,certificaterequest,order,challenge`.

---

## k3s
Expand Down Expand Up @@ -69,37 +114,41 @@ cilium install \
--set operator.replicas=1
cilium status --wait

# 3) TLS + apply the k3s overlay.
# 3) Secrets (above) + install the chart with the k3s values.
kubectl create namespace windep
kubectl -n windep create secret tls windep-api-tls --cert=api.crt --key=api.key
kubectl apply -k platform/overlays/k3s
helm install windep platform/windep -n windep -f platform/windep/values-k3s.yaml
```

Notes:

- **metrics-server**: k3s bundles it (needed by the HPA). If you added `--disable=metrics-server`,
install it separately or the HPA will not scale.
- **Single node**: for a one-node k3s, keep `minReplicas: 2` only if the node can run 2 pods;
otherwise lower it in an overlay patch. Anycast/ECMP is moot on a single node — BGP still
advertises the VIP, just from one path.
- **L2 instead of BGP**: on a flat segment with no BGP peer, swap `cilium/bgp-peering.yaml` for
`cilium/l2announcement.yaml` (via an overlay) and set `--set l2announcements.enabled=true` on the
Cilium install. L2 is failover, not anycast.
- **Single node**: for a one-node k3s, lower `api.hpa.minReplicas` / `api.replicas` if the node
can't run two API pods. Anycast/ECMP is moot on a single node — BGP still advertises the VIP,
just from one path.
- **L2 instead of BGP**: on a flat segment with no BGP peer, set `networking.mode: l2` and install
Cilium with `--set l2announcements.enabled=true`. L2 is failover, not anycast.

---

## The variable file
## Values reference (per-site knobs)

`overlays/<env>/vars.yaml`:
| Value | Into | Notes |
|-------|------|-------|
| `web.vip` / `admin.vip` | Service `io.cilium/lb-ipam-ips` annotation | anycast VIPs |
| `networking.lbPoolCIDR` | `CiliumLoadBalancerIPPool` block | pool the VIPs draw from |
| `networking.advertiseGroup` | `CiliumBGPAdvertisement` `advertise:` label | must match the cluster's peer configs, or the VIP is allocated but never advertised |
| `networking.mode` | `bgp` → advertisement, `l2` → announcement | never both |
| `admin.allowedCIDRs` | `windep-admin` NetworkPolicy ingress | source subnets for the RW VIP |
| `storage.webRoot.storageClass` | payload PVC | must be ReadWriteMany |
| `images.{api,admin,nginx}.tag` | container images | per-site registry/tag |
| `certManager.issuer.acme.*` | cert-manager `Issuer` | internal ACME directory + DNS-01 solver |

| Field | Injected into | Type |
|-------|---------------|------|
| `anycastIP` | Service `io.cilium/lb-ipam-ips` annotation | string |
| `lbPoolCIDR` | `CiliumLoadBalancerIPPool` block | string |
| `advertiseGroup` | `CiliumBGPAdvertisement` `advertise` label | string |
---

`advertiseGroup` must match the label the cluster's `CiliumBGPPeerConfig`s select
(`families[].advertisements.matchLabels.advertise`), or the VIP is allocated but never advertised.
## Network boot & DHCP

The image (registry/tag) is set with the native Kustomize `images:` transformer in each overlay's
`kustomization.yaml`.
Standing up the payloads and wiring UEFI HTTPS Boot / TFTP PXE (DHCP options 60/66/67) is covered
in [../Server/README.md](../Server/README.md). The chart provisions the `windep-web` deploy server
(which serves `/boot`, `/images`, `/config` and proxies `/api/*`); point your DHCP boot URL at that
VIP once the payload PV is populated via the admin console.
18 changes: 0 additions & 18 deletions platform/base/admin/db-pvc.yaml

This file was deleted.

33 changes: 0 additions & 33 deletions platform/base/admin/networkpolicy.yaml

This file was deleted.

27 changes: 0 additions & 27 deletions platform/base/admin/service.yaml

This file was deleted.

33 changes: 0 additions & 33 deletions platform/base/cilium/bgp-advertisement.yaml

This file was deleted.

20 changes: 0 additions & 20 deletions platform/base/cilium/l2announcement.yaml

This file was deleted.

12 changes: 0 additions & 12 deletions platform/base/cilium/lb-ippool.yaml

This file was deleted.

6 changes: 0 additions & 6 deletions platform/base/k8s/namespace.yaml

This file was deleted.

10 changes: 0 additions & 10 deletions platform/base/k8s/poddisruptionbudget.yaml

This file was deleted.

Loading
Loading