Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce InstanceTypes #403

Merged
merged 8 commits into from
Oct 16, 2024
Merged

Introduce InstanceTypes #403

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
4695cd6
Introduce InstanceTypes
kvaps Oct 8, 2024
a31edcc
Split Virtual Machine to vm-instance and vm-disk
kvaps Oct 15, 2024
9d3f278
disable persistence
kvaps Oct 15, 2024
832c94b
refactor templates
kvaps Oct 15, 2024
ca639e6
fix controller revision error
kvaps Oct 15, 2024
6d864ed
refactor
kvaps Oct 16, 2024
f81e343
Add resize hook
kvaps Oct 16, 2024
a851f4b
Merge branch 'main' into instance-types
kvaps Oct 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/apps/kubernetes/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.12.0
version: 0.13.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
Expand Down
191 changes: 173 additions & 18 deletions packages/apps/kubernetes/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,26 +27,181 @@ How to access to deployed cluster:
kubectl get secret -n <namespace> kubernetes-<clusterName>-admin-kubeconfig -o go-template='{{ printf "%s\n" (index .data "super-admin.conf" | base64decode) }}' > test
```

## Parameters
# Series

### Common parameters
<!-- source: https://github.com/kubevirt/common-instancetypes/blob/main/README.md -->

| Name | Description | Value |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
| `host` | The hostname used to access the Kubernetes cluster externally (defaults to using the cluster name as a subdomain for the tenant host). | `""` |
| `controlPlane.replicas` | Number of replicas for Kubernetes contorl-plane components | `2` |
| `storageClass` | StorageClass used to store user data | `replicated` |
| `nodeGroups` | nodeGroups configuration | `{}` |
. | U | O | CX | M | RT
----------------------------|-----|-----|------|-----|------
*Has GPUs* | | | | |
*Hugepages* | | | ✓ | ✓ | ✓
*Overcommitted Memory* | | ✓ | | |
*Dedicated CPU* | | | ✓ | | ✓
*Burstable CPU performance* | ✓ | ✓ | | ✓ |
*Isolated emulator threads* | | | ✓ | | ✓
*vNUMA* | | | ✓ | | ✓
*vCPU-To-Memory Ratio* | 1:4 | 1:4 | 1:2 | 1:8 | 1:4
Comment on lines +30 to +43
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix table formatting for proper Markdown rendering.

The Series table provides valuable information, but its formatting needs adjustment for correct Markdown rendering. Please apply the following changes:

  1. Add a leading pipe (|) to each row.
  2. Ensure all rows have the same number of columns (6 in total).
  3. Add a separator row after the header row.

Here's the corrected table format:

|                            |  U  |  O  |  CX  |  M  |  RT  |
|----------------------------|-----|-----|------|-----|------|
| *Has GPUs*                 |     |     |      |     |      |
| *Hugepages*                |     |     |  ✓   |  ✓  |  ✓   |
| *Overcommitted Memory*     |     |  ✓  |      |     |      |
| *Dedicated CPU*            |     |     |  ✓   |     |  ✓   |
| *Burstable CPU performance*|  ✓  |  ✓  |      |  ✓  |      |
| *Isolated emulator threads*|     |     |  ✓   |     |  ✓   |
| *vNUMA*                    |     |     |  ✓   |     |  ✓   |
| *vCPU-To-Memory Ratio*     | 1:4 | 1:4 |  1:2 | 1:8 | 1:4  |

This will ensure the table is correctly displayed in Markdown viewers.

🧰 Tools
🪛 Markdownlint

36-36: Expected: no_leading_or_trailing; Actual: trailing_only; Unexpected trailing pipe
Table pipe style

(MD055, table-pipe-style)

38-38: Expected: no_leading_or_trailing; Actual: trailing_only; Unexpected trailing pipe
Table pipe style

(MD055, table-pipe-style)

40-40: Expected: no_leading_or_trailing; Actual: trailing_only; Unexpected trailing pipe
Table pipe style

(MD055, table-pipe-style)

36-36: Expected: 6; Actual: 5; Too few cells, row will be missing data
Table column count

(MD056, table-column-count)

38-38: Expected: 6; Actual: 5; Too few cells, row will be missing data
Table column count

(MD056, table-column-count)

40-40: Expected: 6; Actual: 5; Too few cells, row will be missing data
Table column count

(MD056, table-column-count)


### Cluster Addons

| Name | Description | Value |
| ------------------------------------ | ---------------------------------------------------------------------------------- | ------- |
| `addons.certManager.enabled` | Enables the cert-manager | `false` |
| `addons.certManager.valuesOverride` | Custom values to override | `{}` |
| `addons.ingressNginx.enabled` | Enable Ingress-NGINX controller (expect nodes with 'ingress-nginx' role) | `false` |
| `addons.ingressNginx.valuesOverride` | Custom values to override | `{}` |
| `addons.ingressNginx.hosts` | List of domain names that should be passed through to the cluster by upper cluster | `[]` |
| `addons.fluxcd.enabled` | Enables Flux CD | `false` |
| `addons.fluxcd.valuesOverride` | Custom values to override | `{}` |
## U Series

The U Series is quite neutral and provides resources for
general purpose applications.

*U* is the abbreviation for "Universal", hinting at the universal
attitude towards workloads.

VMs of instance types will share physical CPU cores on a
time-slice basis with other VMs.

### U Series Characteristics

Specific characteristics of this series are:
- *Burstable CPU performance* - The workload has a baseline compute
performance but is permitted to burst beyond this baseline, if
excess compute resources are available.
- *vCPU-To-Memory Ratio (1:4)* - A vCPU-to-Memory ratio of 1:4, for less
noise per node.

## O Series

The O Series is based on the U Series, with the only difference
being that memory is overcommitted.

*O* is the abbreviation for "Overcommitted".

### UO Series Characteristics

Specific characteristics of this series are:
- *Burstable CPU performance* - The workload has a baseline compute
performance but is permitted to burst beyond this baseline, if
excess compute resources are available.
- *Overcommitted Memory* - Memory is over-committed in order to achieve
a higher workload density.
- *vCPU-To-Memory Ratio (1:4)* - A vCPU-to-Memory ratio of 1:4, for less
noise per node.

## CX Series

The CX Series provides exclusive compute resources for compute
intensive applications.

*CX* is the abbreviation of "Compute Exclusive".

The exclusive resources are given to the compute threads of the
VM. In order to ensure this, some additional cores (depending
on the number of disks and NICs) will be requested to offload
the IO threading from cores dedicated to the workload.
In addition, in this series, the NUMA topology of the used
cores is provided to the VM.

### CX Series Characteristics

Specific characteristics of this series are:
- *Hugepages* - Hugepages are used in order to improve memory
performance.
- *Dedicated CPU* - Physical cores are exclusively assigned to every
vCPU in order to provide fixed and high compute guarantees to the
workload.
- *Isolated emulator threads* - Hypervisor emulator threads are isolated
from the vCPUs in order to reduce emaulation related impact on the
workload.
- *vNUMA* - Physical NUMA topology is reflected in the guest in order to
optimize guest sided cache utilization.
- *vCPU-To-Memory Ratio (1:2)* - A vCPU-to-Memory ratio of 1:2.

## M Series

The M Series provides resources for memory intensive
applications.

*M* is the abbreviation of "Memory".

### M Series Characteristics

Specific characteristics of this series are:
- *Hugepages* - Hugepages are used in order to improve memory
performance.
- *Burstable CPU performance* - The workload has a baseline compute
performance but is permitted to burst beyond this baseline, if
excess compute resources are available.
- *vCPU-To-Memory Ratio (1:8)* - A vCPU-to-Memory ratio of 1:8, for much
less noise per node.

## RT Series

The RT Series provides resources for realtime applications, like Oslat.

*RT* is the abbreviation for "realtime".

This series of instance types requires nodes capable of running
realtime applications.

### RT Series Characteristics

Specific characteristics of this series are:
- *Hugepages* - Hugepages are used in order to improve memory
performance.
- *Dedicated CPU* - Physical cores are exclusively assigned to every
vCPU in order to provide fixed and high compute guarantees to the
workload.
- *Isolated emulator threads* - Hypervisor emulator threads are isolated
from the vCPUs in order to reduce emaulation related impact on the
workload.
- *vCPU-To-Memory Ratio (1:4)* - A vCPU-to-Memory ratio of 1:4 starting from
the medium size.

## Resources

The following instancetype resources are provided by Cozystack:

Name | vCPUs | Memory
-----|-------|-------
cx1.2xlarge | 8 | 16Gi
cx1.4xlarge | 16 | 32Gi
cx1.8xlarge | 32 | 64Gi
cx1.large | 2 | 4Gi
cx1.medium | 1 | 2Gi
cx1.xlarge | 4 | 8Gi
gn1.2xlarge | 8 | 32Gi
gn1.4xlarge | 16 | 64Gi
gn1.8xlarge | 32 | 128Gi
gn1.xlarge | 4 | 16Gi
m1.2xlarge | 8 | 64Gi
m1.4xlarge | 16 | 128Gi
m1.8xlarge | 32 | 256Gi
m1.large | 2 | 16Gi
m1.xlarge | 4 | 32Gi
n1.2xlarge | 16 | 32Gi
n1.4xlarge | 32 | 64Gi
n1.8xlarge | 64 | 128Gi
n1.large | 4 | 8Gi
n1.medium | 4 | 4Gi
n1.xlarge | 8 | 16Gi
o1.2xlarge | 8 | 32Gi
o1.4xlarge | 16 | 64Gi
o1.8xlarge | 32 | 128Gi
o1.large | 2 | 8Gi
o1.medium | 1 | 4Gi
o1.micro | 1 | 1Gi
o1.nano | 1 | 512Mi
o1.small | 1 | 2Gi
o1.xlarge | 4 | 16Gi
rt1.2xlarge | 8 | 32Gi
rt1.4xlarge | 16 | 64Gi
rt1.8xlarge | 32 | 128Gi
rt1.large | 2 | 8Gi
rt1.medium | 1 | 4Gi
rt1.micro | 1 | 1Gi
rt1.small | 1 | 2Gi
rt1.xlarge | 4 | 16Gi
u1.2xlarge | 8 | 32Gi
u1.2xmedium | 2 | 4Gi
u1.4xlarge | 16 | 64Gi
u1.8xlarge | 32 | 128Gi
u1.large | 2 | 8Gi
u1.medium | 1 | 4Gi
u1.micro | 1 | 1Gi
u1.nano | 1 | 512Mi
u1.small | 1 | 2Gi
u1.xlarge | 4 | 16Gi
25 changes: 25 additions & 0 deletions packages/apps/kubernetes/templates/cluster.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@ spec:
node-role.kubernetes.io/{{ . }}: ""
{{- end }}
spec:
{{- with .group.instanceType }}
instancetype:
kind: VirtualMachineClusterInstancetype
name: {{ . }}
{{- end }}
runStrategy: Always
template:
metadata:
Expand All @@ -26,10 +31,12 @@ spec:
{{- end }}
spec:
domain:
{{- if and .group.resources .group.resources.cpu }}
cpu:
threads: 1
cores: {{ .group.resources.cpu }}
sockets: 1
{{- end }}
devices:
disks:
- name: system
Expand All @@ -43,8 +50,10 @@ spec:
interfaces:
- name: default
bridge: {}
{{- if and .group.resources .group.resources.memory }}
memory:
guest: {{ .group.resources.memory }}
{{- end }}
evictionStrategy: External
volumes:
- name: system
Expand Down Expand Up @@ -176,6 +185,14 @@ spec:
template:
{{- $kubevirtmachinetemplate | nindent 4 }}
---
{{- $instanceType := dict }}
{{- if $group.instanceType }}
{{- $instanceType = (lookup "instancetype.kubevirt.io/v1beta1" "VirtualMachineClusterInstancetype" "" $group.instanceType) }}
{{- if not $instanceType }}
{{- fail (printf "Specified instancetype not exists in cluster: %s" $group.instanceType) }}
{{- end }}
{{- end }}

apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
Expand All @@ -184,8 +201,16 @@ metadata:
annotations:
cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "{{ $group.minReplicas }}"
cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "{{ $group.maxReplicas }}"
{{- if and $group.resources $group.resources.memory }}
capacity.cluster-autoscaler.kubernetes.io/memory: "{{ $group.resources.memory }}"
{{- else }}
capacity.cluster-autoscaler.kubernetes.io/memory: "{{ $instanceType.spec.memory.guest }}"
{{- end }}
{{- if and $group.resources $group.resources.cpu }}
capacity.cluster-autoscaler.kubernetes.io/cpu: "{{ $group.resources.cpu }}"
{{- else }}
capacity.cluster-autoscaler.kubernetes.io/cpu: "{{ $instanceType.spec.cpu.guest }}"
{{- end }}
spec:
clusterName: {{ $.Release.Name }}
template:
Expand Down
8 changes: 5 additions & 3 deletions packages/apps/kubernetes/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,15 @@ nodeGroups:
md0:
minReplicas: 0
maxReplicas: 10
resources:
cpu: 2
memory: 1024Mi
instanceType: "u1.medium"
ephemeralStorage: 20Gi
roles:
- ingress-nginx

resources:
cpu: ""
memory: ""

## @section Cluster Addons
##
addons:
Expand Down
10 changes: 4 additions & 6 deletions packages/apps/versions_map
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ kubernetes 0.9.0 9b6dd19
kubernetes 0.10.0 ac5c38b
kubernetes 0.11.0 4eaca42
kubernetes 0.11.1 4f430a90
kubernetes 0.12.0 HEAD
kubernetes 0.12.0 31a45c4d
kubernetes 0.13.0 HEAD
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider pinning 'kubernetes 0.13.0' to a specific commit hash

While kubernetes 0.12.0 is now pinned to commit 31a45c4d, kubernetes 0.13.0 remains at HEAD. Using HEAD can lead to unpredictable builds due to ongoing changes. Pinning to a specific commit hash ensures consistency and reproducibility.

mysql 0.1.0 f642698
mysql 0.2.0 8b975ff0
mysql 0.3.0 5ca8823
Expand Down Expand Up @@ -73,11 +74,8 @@ tenant 1.2.0 15478a88
tenant 1.3.0 ceefae03
tenant 1.3.1 c56e5769
tenant 1.4.0 HEAD
virtual-machine 0.1.4 f2015d6
virtual-machine 0.1.5 7cd7de7
virtual-machine 0.2.0 5ca8823
virtual-machine 0.3.0 b908400
virtual-machine 0.4.0 HEAD
vm-disk 0.1.0 HEAD
vm-instance 0.1.0 HEAD
Comment on lines +79 to +80
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Recommend specifying commit hashes for 'vm-disk' and 'vm-instance'

Currently, both vm-disk 0.1.0 and vm-instance 0.1.0 are using HEAD. To maintain build stability and prevent unintended changes from affecting the builds, it's advisable to pin these versions to specific commit hashes.

vpn 0.1.0 f642698
vpn 0.2.0 7151424
vpn 0.3.0 HEAD
10 changes: 0 additions & 10 deletions packages/apps/virtual-machine/Makefile
View file
Open in desktop

This file was deleted.

Loading