Skip to content

Commit 7fede07

Browse files
pratik-mahallepratik-mahalle
committed
docs: Add comprehensive documentation for Metal3 Custom Resources
This commit introduces new documentation files for the Metal3 custom resources, including Metal3Data and Metal3DataTemplate, within the CAPM3 user guide. The added docs provide API references, YAML examples, resource relationships, and usage guidance for managing bare metal infrastructure in Kubernetes. Navigation links in SUMMARY.md have been updated accordingly. Fixes #398 Signed-off-by: pratik-mahalle <mahallepratik683@gmail.com>
1 parent 1364510 commit 7fede07

File tree

7 files changed

+767
-1
lines changed

7 files changed

+767
-1
lines changed

‎design/cluster-api-provider-metal3/multi-tenancy_contract.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -218,4 +218,4 @@ the Metal3Machines.
218218
## References
219219

220220
1. [CAPI multi-tenancy
221-
contract](https://cluster-api.sigs.k8s.io/developer/architecture/controllers/multi-tenancy#contract)
221+
contract](https://cluster-api.sigs.k8s.io/developer/providers/contracts/overview.html)

‎docs/user-guide/src/SUMMARY.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@
4444
- [Node Reuse](capm3/node_reuse.md)
4545
- [Pivoting](capm3/pivoting.md)
4646
- [Automated cleaning](capm3/automated_cleaning.md)
47+
- [Data sources](capm3/data_sources.md)
4748
- [Ip-address-manager](ipam/introduction.md)
4849
- [Install Ip-address-manager](ipam/ipam_installation.md)
4950
- [Troubleshooting FAQ](troubleshooting.md)

‎docs/user-guide/src/capm3/custom_resources.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Custom Resources
2+
3+
Cluster API Provider Metal3 (CAPM3) extends the Kubernetes API with several custom
4+
resources that enable the management of bare metal infrastructure through the
5+
Cluster API framework. These resources provide the necessary abstractions for
6+
defining, configuring, and managing bare metal hosts and their associated
7+
metadata.
8+
9+
## Overview
10+
11+
The custom resources in CAPM3 are designed to work together to provide a complete
12+
solution for bare metal cluster management:
13+
14+
- **Metal3DataTemplate**: Defines templates for generating metadata and network
15+
configuration for bare metal hosts
16+
- **Metal3Data**: Represents the rendered data instances created from templates
17+
- **Metal3Machine**: Represents a bare metal machine in the cluster
18+
- **Metal3Cluster**: Represents a bare metal cluster
19+
20+
## Resource Relationships
21+
22+
```mermaid
23+
Metal3Cluster
24+
↓
25+
Metal3MachineTemplate
26+
↓
27+
Metal3Machine
28+
↓
29+
Metal3DataClaim
30+
↓
31+
Metal3Data ← Metal3DataTemplate
32+
```
33+
34+
## Usage Patterns
35+
36+
### Basic Usage
37+
38+
1. Create a Metal3DataTemplate with your desired metadata and network
39+
configuration
40+
2. Reference the template in your Metal3MachineTemplate
41+
3. CAPM3 automatically creates Metal3Data instances and renders the
42+
configuration
43+
44+
### Advanced Usage
45+
46+
- Use IP pools for dynamic IP address allocation
47+
- Configure complex network topologies with bonds, VLANs, and multiple
48+
interfaces
49+
- Implement custom metadata generation based on host properties

‎docs/user-guide/src/capm3/data_sources.md‎

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
# Using CAPM3 Data Sources
2+
3+
## Resource Relationships Overview
4+
5+
Assuming you've deployed Baremetal Operator and CAPM3, you will likely provision
6+
a cluster and control-plane/worker nodes. This process creates several
7+
interconnected resources and this document aims to explain those. As metal3 is
8+
an infrastructure provider for cluster API (CAPI), it necessarily references
9+
also other CAPI resources, however, this document focuses on metal3 resources.
10+
11+
For more details about CAPI resources and to get a big picture, refer to
12+
CAPI [docs](https://cluster-api.sigs.k8s.io/user/concepts). **Please note that the
13+
following CAPI examples are only to illustrate how metal3 resources fit into the
14+
big picture, so for up to date documentation of CAPI resources, refer to the
15+
official docs.**
16+
17+
Visualization of relationships between metal3 resources can be found for example
18+
[here](https://github.com/metal3-io/cluster-api-provider-metal3/issues/1358).
19+
**Note that the graph is not perfect and there can be missing information.**
20+
21+
The example values in this document are from a cluster deployed with
22+
[metal3 dev env](https://github.com/metal3-io/metal3-dev-env).
23+
24+
### 1. `Cluster`
25+
26+
The `Cluster` resource is **CAPI resource** and includes a reference to the control
27+
plane via the `controlPlaneRef` field:
28+
29+
``` yaml
30+
controlPlaneRef:
31+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
32+
kind: KubeadmControlPlane
33+
name: test1
34+
namespace: metal3
35+
````
36+
37+
This object defines the cluster and links it to the control plane configuration.
38+
39+
### 2. `KubeadmControlPlane`
40+
41+
This is the main entry point for configuring the control plane. This **CAPI
42+
resource** directly references CAPM3 resources used to manage the life cycle of
43+
bare metal machines running under the controlplane nodes.
44+
45+
- Refers to a `KubeadmConfigTemplate`, which defines kubeadm configuration
46+
- Refers to a `Metal3MachineTemplate`, which defines the infrastructure for
47+
control plane nodes
48+
49+
``` yaml
50+
machineTemplate:
51+
infrastructureRef:
52+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
53+
kind: Metal3MachineTemplate
54+
name: test1-controlplane
55+
namespace: metal3
56+
```
57+
58+
### 3. `KubeadmConfigTemplate`
59+
60+
`KubeadmConfigTemplate` is a **CAPI resources** and defines the `kubeadm`
61+
configuration to use when initializing new nodes in the cluster. This template
62+
contains multiple important fields, such as (but not limited to)
63+
64+
- `initConfiguration`
65+
- `joinConfiguration`
66+
- `postKubeadmCommands` commands to run after `kubeadm init`
67+
- `preKubeadmCommands` commands to run before `kubeadm init`
68+
69+
Please refer to official docs for more details.
70+
71+
### 4. `Metal3MachineTemplate` → `Metal3Machine`
72+
73+
The `Metal3MachineTemplate` is **metal3 resource** and it is used to generate
74+
`Metal3Machine` resources on demand when `KubeadmControlPlane` or
75+
`MachineDeployment` triggers the creation (for example when scaling up). Each
76+
`Metal3Machine`:
77+
78+
- Is linked to a CAPI `Machine` resource.
79+
- Consumes a `BareMetalHost` BMH instances on the cluster.
80+
- **Refers to a `Metal3Data`** resource which holds specific node configuration
81+
82+
CAPI resource `Machine` is infrastructure agnostic resource. `Metal3Machine` is
83+
metal3 specific resource which is used internally in CAPM3 to represent
84+
machines. `BareMetalHost` represents bare metal server, and hence
85+
`Metal3Machines` occupy those servers.
86+
87+
``` yaml
88+
status:
89+
...
90+
renderedData:
91+
name: test1-controlplane-template-0
92+
namespace: metal3
93+
```
94+
95+
### 5. `Metal3DataTemplate` → `Metal3Data`
96+
97+
The `Metal3DataTemplate` is **metal3 resource** and it is used to
98+
create `Metal3Data` objects. These objects contain machine-specific
99+
configuration, such as:
100+
101+
- Static IP settings
102+
- Network interfaces (NICs)
103+
104+
> **Important**: This is one of the more complex parts of CAPM3
105+
> configuration because CAPM3 does some preprocessing with the networking data.
106+
> With other templates the data is mostly just copied over to deployed resources
107+
> but the networking template supports convenience features like matching NICs
108+
> (see [NIC matching](### NIC matching). Misconfiguration here can lead to
109+
> provisioning errors: `"msg"="Reconciler error" "error"="Failed to create secrets: NIC name not found enp1s0"`.
110+
> For example, see: [CAPM3 Issue #1998](https://github.com/metal3-io/cluster-api-provider-metal3/issues/1998)
111+
112+
The relevant fields in `Metal3Data` are
113+
114+
``` yaml
115+
spec:
116+
claim:
117+
name: test1-jzktd
118+
namespace: metal3
119+
metaData:
120+
name: test1-jzktd-metadata
121+
namespace: metal3
122+
networkData:
123+
name: test1-jzktd-networkdata
124+
namespace: metal3
125+
template:
126+
name: test1-controlplane-template
127+
namespace: metal3
128+
```
129+
130+
- `claim` refers to `Metal3DataClaim`
131+
- `metaData` refers to `test1-jzktd-metadata` named secret
132+
- `networkData` refers to `test1-jzktd-networkdata` named secret
133+
- `template` refers to `Metal3DataTemplate`
134+
135+
Example content of `networkData` secret is
136+
137+
``` yaml
138+
links:
139+
- ethernet_mac_address: 00:fe:8d:20:55:35
140+
id: enp1s0
141+
mtu: 1500
142+
type: phy
143+
- ethernet_mac_address: 00:fe:8d:20:55:36
144+
id: enp2s0
145+
mtu: 1500
146+
type: phy
147+
networks:
148+
- id: externalv4
149+
ip_address: 192.168.111.100
150+
link: enp2s0
151+
netmask: 255.255.255.0
152+
...
153+
```
154+
155+
#### NIC matching
156+
157+
If we continue the example above, the corresponding configuration in
158+
`Metal3DataTemplate` is
159+
160+
``` yaml
161+
networkData:
162+
links:
163+
ethernets:
164+
- id: enp1s0
165+
macAddress:
166+
fromHostInterface: enp1s0
167+
mtu: 1500
168+
type: phy
169+
- id: enp2s0
170+
macAddress:
171+
fromHostInterface: enp2s0
172+
mtu: 1500
173+
type: phy
174+
networks:
175+
ipv4:
176+
- id: externalv4
177+
ipAddressFromIPPool: externalv4-pool
178+
link: enp2s0
179+
...
180+
```
181+
182+
Notice that the `links` are configured by interface ID `fromHostInterface: enp1s0`.
183+
This is an example of preprocessing made by CAPM3.
184+
185+
CAPM3 matches the NICs in the template (or resolves the `fromHostInterface:
186+
enp1s0`) by *reaching out* and reading `BaremetalHost` inspection data.
187+
188+
The `networks` section in the template defines an IP pool from which to take an
189+
IP address (and with which NIC to associate it with). CAPM3 reserves the IP
190+
address by creating an IP claim *for IPAM* and then inserts the IP address into
191+
the secret which is referenced by the `Metal3Data`.
192+
193+
**Important!** CAPM3 utilizes both BMO and IPAM resources to generate the
194+
machine specific data.
195+
196+
## Template vs. Concrete Resource
197+
198+
Each key resource in CAPM3 (and often in CAPI) typically has a matching
199+
`Template` resource. These templates are used to create real resources on
200+
demand, and users generally only need to configure the templates.
201+
202+
Once these templates are set, CAPM3 will render the actual resources during
203+
cluster creation and updates.

‎docs/user-guide/src/capm3/features.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@
55
- [Pivoting](./pivoting.md)
66
- [Automated cleaning](./automated_cleaning.md)
77
- [Label synchronization](./label_sync.md)
8+
- [Data sources](./data_sources.md)

0 commit comments

Comments
 (0)