diff --git a/config.toml b/config.toml index ba0bb0231d1..3c435926649 100644 --- a/config.toml +++ b/config.toml @@ -27,7 +27,7 @@ alpine_js_version = "2.2.1" favicon = "favicon.png" [params.versions] -docs = ["2.5","2.4", "2.3", "2.2", "2.1", "2.0", "1.5", "1.4"] +docs = ["2.6", "2.5", "2.4", "2.3", "2.2", "2.1", "2.0", "1.5", "1.4"] # Site fonts. For more options see https://fonts.google.com. [[params.fonts]] diff --git a/content/docs/2.7/_index.md b/content/docs/2.7/_index.md new file mode 100644 index 00000000000..a3c18da2f7a --- /dev/null +++ b/content/docs/2.7/_index.md @@ -0,0 +1,8 @@ ++++ +title = "The KEDA Documentation" +weight = 1 ++++ + +Welcome to the documentation for **KEDA**, the Kubernetes Event-driven Autoscaler. Use the navigation to the left to learn more about how to use KEDA and its components. + +Additions and contributions to these docs are managed on [the keda-docs GitHub repo](https://github.com/kedacore/keda-docs). diff --git a/content/docs/2.7/concepts/_index.md b/content/docs/2.7/concepts/_index.md new file mode 100644 index 00000000000..1dd5e936601 --- /dev/null +++ b/content/docs/2.7/concepts/_index.md @@ -0,0 +1,48 @@ ++++ +title = "KEDA Concepts" +description = "What KEDA is and how it works" +weight = 1 ++++ + +## What is KEDA? + +**KEDA** is a [Kubernetes](https://kubernetes.io)-based Event Driven Autoscaler. With KEDA, you can drive the scaling of any container in Kubernetes based on the number of events needing to be processed. + +**KEDA** is a single-purpose and lightweight component that can be added into any Kubernetes cluster. KEDA works alongside standard Kubernetes components like the [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) and can extend functionality without overwriting or duplication. With KEDA you can explicitly map the apps you want to use event-driven scale, with other apps continuing to function. This makes KEDA a flexible and safe option to run alongside any number of any other Kubernetes applications or frameworks. + +## How KEDA works + +KEDA performs two key roles within Kubernetes: + +1. **Agent** — KEDA activates and deactivates Kubernetes [Deployments](https://kubernetes.io/docs/concepts/workloads/controllers/deployment) to scale to and from zero on no events. This is one of the primary roles of the `keda-operator` container that runs when you install KEDA. +1. **Metrics** — KEDA acts as a [Kubernetes metrics server](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-custom-metrics) that exposes rich event data like queue length or stream lag to the Horizontal Pod Autoscaler to drive scale out. It is up to the Deployment to consume the events directly from the source. This preserves rich event integration and enables gestures like completing or abandoning queue messages to work out of the box. The metric serving is the primary role of the `keda-operator-metrics-apiserver` container that runs when you install KEDA. + +## Architecture + +The diagram below shows how KEDA works in conjunction with the Kubernetes Horizontal Pod Autoscaler, external event sources, and Kubernetes' [etcd](https://etcd.io) data store: + +![KEDA architecture](/img/keda-arch.png) + +### Event sources and scalers + +KEDA has a wide range of [**scalers**](/scalers) that can both detect if a deployment should be activated or deactivated, and feed custom metrics for a specific event source. The following scalers are available: + +{{< scalers-compact >}} + +### Custom Resources (CRD) + +When you install KEDA, it creates four custom resources: + +1. `scaledobjects.keda.sh` +1. `scaledjobs.keda.sh` +1. `triggerauthentications.keda.sh` +1. `clustertriggerauthentications.keda.sh` + +These custom resources enable you to map an event source (and the authentication to that event source) to a Deployment, StatefulSet, Custom Resource or Job for scaling. +- `ScaledObjects` represent the desired mapping between an event source (e.g. Rabbit MQ) and the Kubernetes Deployment, StatefulSet or any Custom Resource that defines `/scale` subresource. +- `ScaledJobs` represent the mapping between event source and Kubernetes Job. +- `ScaledObject`/`ScaledJob` may also reference a `TriggerAuthentication` or `ClusterTriggerAuthentication` which contains the authentication configuration or secrets to monitor the event source. + +## Deploy KEDA + +See the [Deployment](../deploy) documentation for instructions on how to deploy KEDA into any cluster using tools like [Helm](../deploy/#helm). diff --git a/content/docs/2.7/concepts/authentication.md b/content/docs/2.7/concepts/authentication.md new file mode 100644 index 00000000000..96bb8728399 --- /dev/null +++ b/content/docs/2.7/concepts/authentication.md @@ -0,0 +1,254 @@ ++++ +title = "Authentication" +weight = 500 ++++ + +Often a scaler will require authentication or secrets and config to check for events. + +KEDA provides a few secure patterns to manage authentication flows: + +* Configure authentication per `ScaledObject` +* Re-use per-namespace credentials or delegate authentication with `TriggerAuthentication` +* Re-use global credentials with `ClusterTriggerAuthentication` + +## Defining secrets and config maps on ScaledObject + +Some metadata parameters will not allow resolving from a literal value, and will instead require a reference to a secret, config map, or environment variable defined on the target container. + +> 💡 ***TIP:*** *If creating a deployment yaml that references a secret, be sure the secret is created before the deployment that references it, and the scaledObject after both of them to avoid invalid references.* + +### Example + +If using the [RabbitMQ scaler](https://keda.sh/docs/2.1/scalers/rabbitmq-queue/), the `host` parameter may include passwords so is required to be a reference. You can create a secret with the value of the `host` string, reference that secret in the deployment, and map it to the `ScaledObject` metadata parameter like below: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: {secret-name} +data: + {secret-key-name}: YW1xcDovL3VzZXI6UEFTU1dPUkRAcmFiYml0bXEuZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2NhbDo1Njcy #base64 encoded per secret spec +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {deployment-name} + namespace: default + labels: + app: {deployment-name} +spec: + selector: + matchLabels: + app: {deployment-name} + template: + metadata: + labels: + app: {deployment-name} + spec: + containers: + - name: {deployment-name} + image: {container-image} + envFrom: + - secretRef: + name: {secret-name} +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: {scaled-object-name} + namespace: default +spec: + scaleTargetRef: + name: {deployment-name} + triggers: + - type: rabbitmq + metadata: + queueName: hello + host: {secret-key-name} + queueLength : '5' +``` + +If you have multiple containers in a deployment, you will need to include the name of the container that has the references in the `ScaledObject`. If you do not include a `envSourceContainerName` it will default to the first container. KEDA will attempt to resolve references from secrets, config maps, and environment variables of the container. + +### The downsides + +While this method works for many scenarios, there are some downsides: + +* **Difficult to efficiently share auth** config across `ScaledObjects` +* **No support for referencing a secret directly**, only secrets that are referenced by the container +* **No support for other types of authentication flows** such as *pod identity* where access to a source could be acquired with no secrets or connection strings + +For these and other reasons, we also provide a `TriggerAuthentication` resource to define authentication as a separate resource to a `ScaledObject`. This allows you to reference secrets directly, configure to use pod identity or use authentication object managed by a different team. + +## Re-use credentials and delegate auth with TriggerAuthentication + +`TriggerAuthentication` allows you to describe authentication parameters separate from the `ScaledObject` and the deployment containers. It also enables more advanced methods of authentication like "pod identity", authentication re-use or allowing IT to configure the authentication. + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: {trigger-authentication-name} + namespace: default # must be same namespace as the ScaledObject +spec: + podIdentity: + provider: none | azure | aws-eks | aws-kiam # Optional. Default: none + secretTargetRef: # Optional. + - parameter: {scaledObject-parameter-name} # Required. + name: {secret-name} # Required. + key: {secret-key-name} # Required. + env: # Optional. + - parameter: {scaledObject-parameter-name} # Required. + name: {env-name} # Required. + containerName: {container-name} # Optional. Default: scaleTargetRef.envSourceContainerName of ScaledObject + hashiCorpVault: # Optional. + address: {hashicorp-vault-address} # Required. + namespace: {hashicorp-vault-namespace} # Optional. Default is root namespace. Useful for Vault Enterprise + authentication: token | kubernetes # Required. + role: {hashicorp-vault-role} # Optional. + mount: {hashicorp-vault-mount} # Optional. + credential: # Optional. + token: {hashicorp-vault-token} # Optional. + serviceAccount: {path-to-service-account-file} # Optional. + secrets: # Required. + - parameter: {scaledObject-parameter-name} # Required. + key: {hasicorp-vault-secret-key-name} # Required. + path: {hasicorp-vault-secret-path} # Required. +``` + +Based on the requirements you can mix and match the reference types providers in order to configure all required parameters. + +Every parameter you define in `TriggerAuthentication` definition does not need to be included in the `metadata` of the trigger for your `ScaledObject` definition. To reference a `TriggerAuthentication` from a `ScaledObject` you add the `authenticationRef` to the trigger. + +```yaml +# some Scaled Object +# ... + triggers: + - type: {scaler-type} + metadata: + param1: {some-value} + authenticationRef: + name: {trigger-authentication-name} # this may define other params not defined in metadata +``` + +## Authentication scopes: Namespace vs. Cluster + +Each `TriggerAuthentication` is defined in one namespace and can only be used by a `ScaledObject` in that same namespace. For cases where you want to share a single set of credentials between scalers in many namespaces, you can instead create a `ClusterTriggerAuthentication`. As a global object, this can be used from any namespace. To set a trigger to use a `ClusterTriggerAuthentication`, add a `kind` field to the authentication reference: + +```yaml + authenticationRef: + name: {cluster-trigger-authentication-name} + kind: ClusterTriggerAuthentication +``` + +By default, Secrets loaded from a `secretTargetRef` must be in the same namespace as KEDA is deployed in (usually `keda`). This can be overridden by setting a `KEDA_CLUSTER_OBJECT_NAMESPACE` environment variable for the `keda-operator` container. + +Defining a `ClusterTriggerAuthentication` works almost identically to a `TriggerAuthentication`, except there is no `metadata.namespace` value: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ClusterTriggerAuthentication +metadata: + name: {cluster-trigger-authentication-name} +spec: + # As before ... +``` + +## Authentication parameters + +Authentication parameters can be pulled in from many sources. All of these values are merged together to make the authentication data for the scaler. + +### Environment variable(s) + +You can pull information via one or more environment variables by providing the `name` of the variable for a given `containerName`. + +```yaml +env: # Optional. + - parameter: region # Required - Defined by the scale trigger + name: my-env-var # Required. + containerName: my-container # Optional. Default: scaleTargetRef.envSourceContainerName of ScaledObject +``` + +**Assumptions:** `containerName` is in the same resource as referenced by `scaleTargetRef.name` in the ScaledObject, unless specified otherwise. + +### Secret(s) + +You can pull one or more secrets into the trigger by defining the `name` of the Kubernetes Secret and the `key` to use. + +```yaml +secretTargetRef: # Optional. + - parameter: connectionString # Required - Defined by the scale trigger + name: my-keda-secret-entity # Required. + key: azure-storage-connectionstring # Required. +``` + +**Assumptions:** `namespace` is in the same resource as referenced by `scaleTargetRef.name` in the ScaledObject, unless specified otherwise. + +### Hashicorp Vault secret(s) + +You can pull one or more Hashicorp Vault secrets into the trigger by defining the authentication metadata such as Vault `address` and the `authentication` method (token | kubernetes). If you choose kubernetes auth method you should provide `role` and `mount` as well. +`credential` defines the Hashicorp Vault credentials depending on the authentication method, for kubernetes you should provide path to service account token (/var/run/secrets/kubernetes.io/serviceaccount/token) and for token auth method provide the token. +`secrets` list defines the mapping between the path and the key of the secret in Vault to the parameter. +`namespace` may be used to target a given Vault Enterprise namespace. + +```yaml +hashiCorpVault: # Optional. + address: {hashicorp-vault-address} # Required. + namespace: {hashicorp-vault-namespace} # Optional. Default is root namespace. Useful for Vault Enterprise + authentication: token | kubernetes # Required. + role: {hashicorp-vault-role} # Optional. + mount: {hashicorp-vault-mount} # Optional. + credential: # Optional. + token: {hashicorp-vault-token} # Optional. + serviceAccount: {path-to-service-account-file} # Optional. + secrets: # Required. + - parameter: {scaledObject-parameter-name} # Required. + key: {hasicorp-vault-secret-key-name} # Required. + path: {hasicorp-vault-secret-path} # Required. +``` + +### Pod Authentication Providers + +Several service providers allow you to assign an identity to a pod. By using that identity, you can defer authentication to the pod & the service provider, rather than configuring secrets. + +Currently we support the following: + +```yaml +podIdentity: + provider: none | azure | aws-eks | aws-kiam # Optional. Default: none +``` + +#### Azure Pod Identity + +Azure Pod Identity is an implementation of [**Azure AD Pod Identity**](https://github.com/Azure/aad-pod-identity) which let's you bind an [**Azure Managed Identity**](https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/) to a Pod in a Kubernetes cluster as delegated access - *Don't manage secrets, let Azure AD do the hard work*. + +You can tell KEDA to use Azure AD Pod Identity via `podIdentity.provider`. + +```yaml +podIdentity: + provider: azure # Optional. Default: false +``` + +Azure AD Pod Identity will give access to containers with a defined label for `aadpodidbinding`. You can set this label on the KEDA operator deployment. This can be done for you during deployment with Helm with `--set podIdentity.activeDirectory.identity={your-label-name}`. + +#### EKS Pod Identity Webhook for AWS + +[**EKS Pod Identity Webhook**](https://github.com/aws/amazon-eks-pod-identity-webhook), which is described more in depth [here](https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/), allows you to provide the role name using an annotation on a service account associated with your pod. + +You can tell KEDA to use EKS Pod Identity Webhook via `podIdentity.provider`. + +```yaml +podIdentity: + provider: aws-eks # Optional. Default: false +``` + +#### Kiam Pod Identity for AWS + +[**Kiam**](https://github.com/uswitch/kiam/) lets you bind an AWS IAM Role to a pod using an annotation on the pod. + +You can tell KEDA to use Kiam via `podIdentity.provider`. + +```yaml +podIdentity: + provider: aws-kiam # Optional. Default: false +``` diff --git a/content/docs/2.7/concepts/external-scalers.md b/content/docs/2.7/concepts/external-scalers.md new file mode 100644 index 00000000000..2574b8d5041 --- /dev/null +++ b/content/docs/2.7/concepts/external-scalers.md @@ -0,0 +1,630 @@ ++++ +title = "External Scalers" +weight = 500 ++++ + +While KEDA ships with a set of [built-in scalers](../scalers), users can also extend KEDA through a [GRPC](https://grpc.io) service that implements the same interface as the built-in scalers. + +Built-in scalers run in the KEDA process/pod, while external scalers require an externally managed GRPC server that's accessible from KEDA with optional [TLS authentication](https://grpc.io/docs/guides/auth/). KEDA itself acts as a GRPC client and it exposes similar service interface for the built-in scalers, so external scalers can fully replace built-in ones. + +This document describes the external scaler interfaces and how to implement them in Go, Node, and .NET; however for more details on GRPC refer to [the official GRPC documentation](https://grpc.io/docs/) + +>Want to learn about existing external scalers? Explore our [external scaler community](https://github.com/kedacore/external-scalers). + +## Overview + +### Built-in scalers interface + +Since external scalers mirror the interface of built-in scalers, it's worth becoming familiar with the Go `interface` that the built-in scalers implement: + +```go +type Scaler interface { + GetMetrics(ctx context.Context, metricName string, metricSelector labels.Selector) ([]external_metrics.ExternalMetricValue, error) + GetMetricSpecForScaling() []v2beta2.MetricSpec + IsActive(ctx context.Context) (bool, error) + Close() error +} + +type PushScaler interface { + Scaler + + Run(ctx context.Context, active chan<- bool) +} +``` + +The `Scaler` interface defines 4 methods: + +- `IsActive` is called on `pollingInterval` defined in the ScaledObject/ScaledJob CRDs. KEDA will scale to 1 if this method returns `true`. +- `Close` is called to allow the scaler to clean up connections or other resources. +- `GetMetricSpec` returns the target value for the HPA definition for the scaler. For more details refer to [Implementing `GetMetricSpec`](#5-implementing-getmetricspec). +- `GetMetrics` returns the value of the metric referred to from `GetMetricSpec`. For more details refer to [Implementing `GetMetrics`](#6-implementing-getmetrics). +> Refer to the [HPA docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/) for how HPA calculates `replicaCount` based on metric value and target value. KEDA uses the metric target type `AverageValue` for external metrics. This will cause the metric value returned by the external scaler to be divided by the number of replicas. + +The `PushScaler` interface adds a `Run` method. This method receives a push channel (`active`), on which the scaler can send `true` at any time. The purpose of this mechanism is to initiate a scaling operation independently from `pollingInterval`. + +### External Scaler GRPC interface + +KEDA comes with 2 external scalers [`external`](../scalers/external.md) and [`external-push`](../scalers/external-push.md). + +The configuration in the ScaledObject points to a GRPC service endpoint that implements the [`externalscaler.proto`](https://github.com/kedacore/keda/blob/main/pkg/scalers/externalscaler/externalscaler.proto) GRPC contract: + +```proto +service ExternalScaler { + rpc IsActive(ScaledObjectRef) returns (IsActiveResponse) {} + rpc StreamIsActive(ScaledObjectRef) returns (stream IsActiveResponse) {} + rpc GetMetricSpec(ScaledObjectRef) returns (GetMetricSpecResponse) {} + rpc GetMetrics(GetMetricsRequest) returns (GetMetricsResponse) {} +} +``` + +Much of this contract is similar to the built-in scalers: + +- `GetMetrics` and `GetMetricsSpec` mirror their counterparts in the `Scaler` interface for creating HPA definition. +- `IsActive` maps to the `IsActive` method on the `Scaler` interface. +- `StreamIsActive` maps to the `Run` method on the `PushScaler` interface. + +There are, however, some notable differences: + +- There is no `Close` method. The scaler is expected to be functional throughout its lifetime. +- `IsActive`, `StreamIsActive`, and `GetMetricsSpec` are called with a `ScaledObjectRef` that contains the scaledObject name/namespace as well as the content of `metadata` defined in the trigger. + +### Example + +Given the following `ScaledObject`: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: scaledobject-name + namespace: scaledobject-namespace +spec: + scaleTargetRef: + name: deployment-name + triggers: + - type: external-push + metadata: + scalerAddress: service-address.svc.local:9090 + key1: value1 + key2: value2 +``` + +KEDA will attempt a GRPC connection to `service-address.svc.local:9090` immediately after reconciling the `ScaledObject`. It will then make the following RPC calls: + +- `IsActive` - KEDA does an initial call to `IsActive` followed by one call on each `pollingInterval` +- `StreamIsActive` - KEDA does an initial call and the scaler is expected to maintain a long-lived connection (called a `stream` in GRPC terminology). The external push scaler can then send an `IsActive` event back to KEDA at any time. KEDA will only attempt another call to `StreamIsActive` if it needs to re-connect +- `GetMetricsSpec` - KEDA will do an initial call with the following data in the incoming `ScaledObjectRef` parameter: +- `GetMetrics` - KEDA will call this method every `pollingInterval` to get the point-in-time metric values for the names returned by `GetMetricsSpec`. + +```json +{ + "name": "scaledobject-name", + "namespace": "scaledobject-namespace", + "scalerMetadata": { + "scalerAddress": "service-address.svc.local:9090", + "key1": "value1", + "key2": "value2" + } +} +``` + +>**Note**: KEDA will issue all of the above RPC calls except `StreamIsActive` if `spec.triggers.type` is `external`. It _must_ be `external-push` for `StreamIsActive` to be called. + +## Implementing KEDA external scaler GRPC interface + +### Implementing an external scaler + +#### 1. Download [`externalscaler.proto`](https://github.com/kedacore/keda/blob/main/pkg/scalers/externalscaler/externalscaler.proto) + +#### 2. Prepare project + +{{< collapsible "Golang" >}} + +2.1. Download [`./protoc`](https://github.com/protocolbuffers/protobuf/releases) for your platform + +2.2. get `protoc-gen-go` + +```bash +go get github.com/golang/protobuf/protoc-gen-go@v1.3.2 +``` + +2.3. Prepare project + +```bash +go mod init example.com/external-scaler/sample +mkdir externalscaler +protoc externalscaler.proto --go_out=plugins=grpc:externalscaler +``` + +{{< /collapsible >}} + +{{< collapsible "C#" >}} +2.1. Create a new project + +```bash +dotnet new console -o ExternalScalerSample +cd ExternalScalerSample + +# add Grpc.AspNetCore +dotnet add package Grpc.AspNetCore +dotnet add package Newtonsoft.Json + +# Create a Protos and Services folders +mkdir Protos +mkdir Services +``` + +2.2. Move `externalscaler.proto` to `Protos` folder + +2.3. Compile `externalscaler.proto` using this in `ExternalScalerSample.csproj` + +```xml + + + +``` + +{{< /collapsible >}} + +{{< collapsible "Javascript" >}} +2.1. Prepare the project + +```bash +npm install --save grpc request +``` + +{{< /collapsible >}} + +
+ +#### 3. Implementing `IsActive` + +Just like `IsActive(ctx context.Context) (bool, error)` in the go interface, the `IsActive` method in the GRPC interface is called every `pollingInterval` with a `ScaledObjectRef` object that contains the scaledObject name, namespace, and scaler metadata. + +This section implements an external scaler that queries earthquakes from [earthquake.usgs.gov](https://earthquake.usgs.gov/) and scales the deployment if there has been more than 2 earthquakes with `magnitude > 1.0` around a particular longitude/latitude in the previous day. + +Submit the following `ScaledObject` to tell KEDA to start making RPC calls to your external scaler (modifying appropriate fields as necessary): + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: scaledobject-name + namespace: scaledobject-namespace +spec: + scaleTargetRef: + name: deployment-name + triggers: + - type: external + metadata: + scalerAddress: earthquake-scaler:9090 + longitude: "-122.335167" + latitude: "47.608013" +``` + +{{< collapsible "Golang" >}} + +The full implementation can be found at [github.com/kedacore/external-scaler-samples](https://github.com/kedacore/external-scaler-samples). + +Put the following code into your `main.go` file: + +```golang +func (e *ExternalScaler) IsActive(ctx context.Context, scaledObject *pb.ScaledObjectRef) (*pb.IsActiveResponse, error) { + // request.Scalermetadata contains the `metadata` defined in the ScaledObject + longitude := scaledObject.ScalerMetadata["longitude"] + latitude := scaledObject.ScalerMetadata["latitude"] + + if len(longitude) == 0 || len(latitude) == 0 { + return nil, status.Error(codes.InvalidArgument, "longitude and latitude must be specified") + } + + startTime := time.Now().AddDate(0, 0, -1).Format("2006-01-02") + endTime := time.Now().Format("2006-01-02") + radiusKM := 500 + query := fmt.Sprintf("format=geojson&starttime=%s&endtime=%s&longitude=%s&latitude=%s&maxradiuskm=%d", startTime, endTime, longitude, latitude, radiusKM) + + resp, err := http.Get(fmt.Sprintf("https://earthquake.usgs.gov/fdsnws/event/1/query?%s", query)) + if err != nil { + return nil, status.Error(codes.Internal, err.Error()) + } + + defer resp.Body.Close() + body, err := ioutil.ReadAll(resp.Body) + if err != nil { + return nil, status.Error(codes.Internal, err.Error()) + } + + payload := USGSResponse{} + err = json.Unmarshal(body, &payload) + if err != nil { + return nil, status.Error(codes.Internal, err.Error()) + } + + // count how many earthquakes with mag > 1.0 + count := 0 + for _, f := range payload.Features { + if f.Properties.Mag > 1.0 { + count++ + } + } + + // return true if there is more than 2 + return &pb.IsActiveResponse{ + Result: count > 2, + }, nil +} +``` + +{{< /collapsible >}} + +{{< collapsible "C#" >}} + +Full implementation can be found at [github.com/kedacore/external-scaler-samples](https://github.com/kedacore/external-scaler-samples). + +Put the following code into your `Services/ExternalScalerService.cs` file: + +```csharp +public class ExternalScalerService : ExternalScaler.ExternalScalerBase +{ + private static readonly HttpClient _client = new HttpClient(); + + public override async Task IsActive(ScaledObjectRef request, ServerCallContext context) + { + // request.Scalermetadata contains the `metadata` defined in the ScaledObject + if (!request.ScalerMetadata.ContainsKey("latitude") || + !request.ScalerMetadata.ContainsKey("longitude")) { + throw new ArgumentException("longitude and latitude must be specified"); + } + + var longitude = request.ScalerMetadata["longitude"]; + var latitude = request.ScalerMetadata["latitude"]; + var startTime = DateTime.UtcNow.AddDays(-1).ToString("yyyy-MM-dd"); + var endTime = DateTime.UtcNow.ToString("yyyy-MM-dd"); + var radiusKm = 500; + var query = $"format=geojson&starttime={startTime}&endtime={endTime}&longitude={longitude}&latitude={latitude}&maxradiuskm={radiusKm}"; + + var resp = await _client.GetAsync($"https://earthquake.usgs.gov/fdsnws/event/1/query?{query}"); + resp.EnsureSuccessStatusCode(); + var payload = JsonConvert.DeserializeObject(await resp.Content.ReadAsStringAsync()); + + return new IsActiveResponse + { + // return true if there is more than 2 Earthquakes with mag > 1.0 + Result = payload.features.Count(f => f.properties.mag > 1.0) > 2 + }; + } +} +``` + +{{< /collapsible >}} + +{{< collapsible "Javascript" >}} + +Put the following code into your `index.js` file: + +```js +const grpc = require('grpc') +const request = require('request') +const externalScalerProto = grpc.load('externalscaler.proto') + +const server = new grpc.Server() +server.addService(externalScalerProto.externalscaler.ExternalScaler.service, { + isActive: (call, callback) => { + const longitude = call.request.scalerMetadata.longitude + const latitude = call.request.scalerMetadata.latitude + if (!longitude || !latitude) { + callback({ + code: grpc.status.INVALID_ARGUMENT, + details: 'longitude and latitude must be specified', + }) + } else { + const now = new Date() + const yesterday = new Date(new Date().setDate(new Date().getDate()-1)); + + const startTime = `${yesterday.getUTCFullYear()}-${yesterday.getUTCMonth()}-${yesterday.getUTCDay()}` + const endTime = `${now.getUTCFullYear()}-${now.getUTCMonth()}-${now.getUTCDay()}` + const radiusKm = 500 + const query = `format=geojson&starttime=${startTime}&endtime=${endTime}&longitude=${longitude}&latitude=${latitude}&maxradiuskm=${radiusKm}` + + request.get({ + url: `https://earthquake.usgs.gov/fdsnws/event/1/query?${query}`, + json: true, + }, (err, resp, data) => { + if (err) { + callback({ + code: grpc.status.INTERNAL, + details: err, + }) + } else if (resp.statusCode !== 200) { + callback({ + code: grpc.status.INTERNAL, + details: `expected status 200, got ${resp.statusCode}` + }) + } else { + // count how many earthquakes with mag > 1.0 + let count = 0 + data.features.forEach(i => { + if (i.properties.mag > 1.0) { + count++ + } + }) + callback(null, { + result: count > 2, + }) + } + }) + } + } +}) + +server.bind('127.0.0.1:9090', grpc.ServerCredentials.createInsecure()) +console.log('Server listening on 127.0.0.1:9090') + +server.start() +``` + +{{< /collapsible >}} + +
+ +#### 4. Implementing `StreamIsActive` + +Unlike `IsActive`, `StreamIsActive` is called once when KEDA reconciles the `ScaledObject`, and expects the external scaler to maintain a long-lived connection and push `IsActiveResponse` objects whenever the scaler needs KEDA to activate the deployment. + +This implementation creates a timer and queries USGS APIs on that timer, effectively ignoring `pollingInterval` set in the scaledObject. Alternatively any other asynchronous event can be used instead of a timer, like an HTTP request, or a network connection. + +{{< collapsible "Golang" >}} + +```golang +func (e *ExternalScaler) StreamIsActive(scaledObject *pb.ScaledObjectRef, epsServer pb.ExternalScaler_StreamIsActiveServer) error { + longitude := scaledObject.ScalerMetadata["longitude"] + latitude := scaledObject.ScalerMetadata["latitude"] + + if len(longitude) == 0 || len(latitude) == 0 { + return status.Error(codes.InvalidArgument, "longitude and latitude must be specified") + } + + for { + select { + case <-epsServer.Context().Done(): + // call cancelled + return nil + case <-time.Tick(time.Hour * 1): + earthquakeCount, err := getEarthQuakeCount(longitude, latitude) + if err != nil { + // log error + } else if earthquakeCount > 2 { + err = epsServer.Send(&pb.IsActiveResponse{ + Result: true, + }) + } + } + } +} +``` + +{{< /collapsible >}} + +{{< collapsible "C#" >}} + +```csharp +public override Task StreamIsActive(ScaledObjectRef request, IServerStreamWriter responseStream, ServerCallContext context) +{ + if (!request.ScalerMetadata.ContainsKey("latitude") || + !request.ScalerMetadata.ContainsKey("longitude")) + { + throw new ArgumentException("longitude and latitude must be specified"); + } + + var longitude = request.ScalerMetadata["longitude"]; + var latitude = request.ScalerMetadata["latitude"]; + var key = $"{longitude}|{latitude}"; + + while (!context.CancellationToken.IsCancellationRequested) + { + var earthquakeCount = await GetEarthQuakeCount(longitude, latitude); + if (earthquakeCount > 2) { + await responseStream.WriteAsync(new IsActiveResponse + { + Result = true + }); + } + + await Task.Delay(TimeSpan.FromHours(1)); + } +} +``` + +{{< /collapsible >}} + +{{< collapsible "Javascript" >}} + +```js +server.addService(externalScalerProto.externalscaler.ExternalScaler.service, { + // ... + streamIsActive: (call, callback) => { + const longitude = call.request.scalerMetadata.longitude + const latitude = call.request.scalerMetadata.latitude + if (!longitude || !latitude) { + callback({ + code: grpc.status.INVALID_ARGUMENT, + details: 'longitude and latitude must be specified', + }) + } else { + const interval = setInterval(() => { + getEarthquakeCount((err, count) => { + if (err) { + console.error(err) + } else if (count > 2) { + call.write({ + result: true, + }) + } + }) + }, 1000 * 60 * 60); + + call.on('end', () => { + clearInterval(interval) + }) + } + } +}) +``` + +{{< /collapsible >}} + +
+ +#### 5. Implementing `GetMetricSpec` + +`GetMetricSpec` returns the `target` value for [the HPA definition for the scaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/). This scaler will define a static target of 10, but the threshold value is often specified in the metadata for other scalers. + +{{< collapsible "Golang" >}} + +```golang +func (e *ExternalScaler) GetMetricSpec(context.Context, *pb.ScaledObjectRef) (*pb.GetMetricSpecResponse, error) { + return &pb.GetMetricSpecResponse{ + MetricSpecs: []*pb.MetricSpec{{ + MetricName: "earthquakeThreshold", + TargetSize: 10, + }}, + }, nil +} +``` + +{{< /collapsible >}} + +{{< collapsible "C#" >}} + +```csharp +public override Task GetMetricSpec(ScaledObjectRef request, ServerCallContext context) +{ + var resp = new GetMetricSpecResponse(); + + resp.MetricSpecs.Add(new MetricSpec + { + MetricName = "earthquakeThreshold", + TargetSize = 10 + }); + + return Task.FromResult(resp); +} +``` + +{{< /collapsible >}} + +{{< collapsible "Javascript" >}} + +```js +server.addService(externalScalerProto.externalscaler.ExternalScaler.service, { + // ... + getMetricSpec: (call, callback) => { + callback(null, { + metricSpecs: [{ + metricName: 'earthquakeThreshold', + targetSize: 10, + }] + }) + } +}) +``` + +{{< /collapsible >}} + +
+ +#### 6. Implementing `GetMetrics` + +`GetMetrics` returns the value of the metric referred to from `GetMetricSpec`, in this example it's `earthquakeThreshold`. + +{{< collapsible "Golang" >}} + +```golang +func (e *ExternalScaler) GetMetrics(_ context.Context, metricRequest *pb.GetMetricsRequest) (*pb.GetMetricsResponse, error) { + longitude := metricRequest.ScaledObjectRef.ScalerMetadata["longitude"] + latitude := metricRequest.ScaledObjectRef.ScalerMetadata["latitude"] + + if len(longitude) == 0 || len(latitude) == 0 { + return nil, status.Error(codes.InvalidArgument, "longitude and latitude must be specified") + } + + earthquakeCount, err := getEarthQuakeCount(longitude, latitude, 1.0) + if err != nil { + return nil, status.Error(codes.Internal, err.Error()) + } + + return &pb.GetMetricsResponse{ + MetricValues: []*pb.MetricValue{{ + MetricName: "earthquakeThreshold", + MetricValue: int64(earthquakeCount), + }}, + }, nil +} +``` + +{{< /collapsible >}} + +{{< collapsible "C#" >}} + +```csharp +public override async Task GetMetrics(GetMetricsRequest request, ServerCallContext context) +{ + if (!request.ScaledObjectRef.ScalerMetadata.ContainsKey("latitude") || + !request.ScaledObjectRef.ScalerMetadata.ContainsKey("longitude")) + { + throw new ArgumentException("longitude and latitude must be specified"); + } + + var longitude = request.ScaledObjectRef.ScalerMetadata["longitude"]; + var latitude = request.ScaledObjectRef.ScalerMetadata["latitude"]; + + var earthquakeCount = await GetEarthQuakeCount(longitude, latitude); + + var resp = new GetMetricsResponse(); + resp.MetricValues.Add(new MetricValue + { + MetricName = "earthquakeThreshold", + MetricValue_ = earthquakeCount + }); + + return resp; +} +``` + +{{< /collapsible >}} + +{{< collapsible "Javascript" >}} + +```js +server.addService(externalScalerProto.externalscaler.ExternalScaler.service, { + // ... + getMetrics: (call, callback) => { + const longitude = call.request.scaledObjectRef.scalerMetadata.longitude + const latitude = call.request.scaledObjectRef.scalerMetadata.latitude + if (!longitude || !latitude) { + callback({ + code: grpc.status.INVALID_ARGUMENT, + details: 'longitude and latitude must be specified', + }) + } else { + getEarthquakeCount((err, count) => { + if (err) { + callback({ + code: grpc.status.INTERNAL, + details: err, + }) + } else { + callback(null, { + metricValues: [{ + metricName: 'earthquakeThreshold', + metricValue: count, + }] + }) + } + }) + } + } +}) +``` + +{{< /collapsible >}} diff --git a/content/docs/2.7/concepts/scaling-deployments.md b/content/docs/2.7/concepts/scaling-deployments.md new file mode 100644 index 00000000000..194d0be84af --- /dev/null +++ b/content/docs/2.7/concepts/scaling-deployments.md @@ -0,0 +1,213 @@ ++++ +title = "Scaling Deployments, StatefulSets & Custom Resources" +weight = 200 ++++ + +## Overview + +### Scaling of Deployments and StatefulSets + +Deployments and StatefulSets are the most common way to scale workloads with KEDA. + +It allows you to define the Kubernetes Deployment or StatefulSet that you want KEDA to scale based on a scale trigger. KEDA will monitor that service and based on the events that occur it will automatically scale your resource out/in accordingly. + +Behind the scenes, KEDA acts to monitor the event source and feed that data to Kubernetes and the HPA (Horizontal Pod Autoscaler) to drive rapid scale of a resource. Each replica of a resource is actively pulling items from the event source. With KEDA and scaling Deployments/StatefulSet you can scale based on events while also preserving rich connection and processing semantics with the event source (e.g. in-order processing, retries, deadletter, checkpointing). + +For example, if you wanted to use KEDA with an Apache Kafka topic as event source, the flow of information would be: + +* When no messages are pending processing, KEDA can scale the deployment to zero. +* When a message arrives, KEDA detects this event and activates the deployment. +* When the deployment starts running, one of the containers connects to Kafka and starts pulling messages. +* As more messages arrive on the Kafka Topic, KEDA can feed this data to the HPA to drive scale out. +* Each replica of the deployment is actively processing messages. Very likely, each replica is processing a batch of messages in a distributed manner. + +### Scaling of Custom Resources + +With KEDA you can scale any workload defined as any `Custom Resource` (for example `ArgoRollout` [resource](https://argoproj.github.io/argo-rollouts/)). The scaling behaves the same way as scaling for arbitrary Kubernetes `Deployment` or `StatefulSet`. + +The only constraint is that the target `Custom Resource` must define `/scale` [subresource](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#scale-subresource). + +## ScaledObject spec + +This specification describes the `ScaledObject` Custom Resource definition which is used to define how KEDA should scale your application and what the triggers are. The `.spec.ScaleTargetRef` section holds the reference to the target resource, ie. `Deployment`, `StatefulSet` or `Custom Resource`. + +[`scaledobject_types.go`](https://github.com/kedacore/keda/blob/main/api/v1alpha1/scaledobject_types.go) + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: {scaled-object-name} +spec: + scaleTargetRef: + apiVersion: {api-version-of-target-resource} # Optional. Default: apps/v1 + kind: {kind-of-target-resource} # Optional. Default: Deployment + name: {name-of-target-resource} # Mandatory. Must be in the same namespace as the ScaledObject + envSourceContainerName: {container-name} # Optional. Default: .spec.template.spec.containers[0] + pollingInterval: 30 # Optional. Default: 30 seconds + cooldownPeriod: 300 # Optional. Default: 300 seconds + idleReplicaCount: 0 # Optional. Must be less than minReplicaCount + minReplicaCount: 1 # Optional. Default: 0 + maxReplicaCount: 100 # Optional. Default: 100 + fallback: # Optional. Section to specify fallback options + failureThreshold: 3 # Mandatory if fallback section is included + replicas: 6 # Mandatory if fallback section is included + advanced: # Optional. Section to specify advanced options + restoreToOriginalReplicaCount: true/false # Optional. Default: false + horizontalPodAutoscalerConfig: # Optional. Section to specify HPA related options + behavior: # Optional. Use to modify HPA's scaling behavior + scaleDown: + stabilizationWindowSeconds: 300 + policies: + - type: Percent + value: 100 + periodSeconds: 15 + triggers: + # {list of triggers to activate scaling of the target resource} +``` + +> 💡 **NOTE:** You can find all supported triggers [here](/scalers). + +### Details +```yaml + scaleTargetRef: + apiVersion: {api-version-of-target-resource} # Optional. Default: apps/v1 + kind: {kind-of-target-resource} # Optional. Default: Deployment + name: {name-of-target-resource} # Mandatory. Must be in the same namespace as the ScaledObject + envSourceContainerName: {container-name} # Optional. Default: .spec.template.spec.containers[0] +``` + +The reference to the resource this ScaledObject is configured for. This is the resource KEDA will scale up/down and setup an HPA for, based on the triggers defined in `triggers:`. + +To scale Kubernetes Deployments only `name` is needed to be specified, if one wants to scale a different resource such as StatefulSet or Custom Resource (that defines `/scale` subresource), appropriate `apiVersion` (following standard Kubernetes convention, ie. `{api}/{version}`) and `kind` need to be specified. + +`envSourceContainerName` is an optional property that specifies the name of container in the target resource, from which KEDA should try to get environment properties holding secrets etc. If it is not defined, KEDA will try to get environment properties from the first Container, ie. from `.spec.template.spec.containers[0]`. + +**Assumptions:** Resource referenced by `name` (and `apiVersion`, `kind`) is in the same namespace as the ScaledObject + +--- + +```yaml + pollingInterval: 30 # Optional. Default: 30 seconds +``` + +This is the interval to check each trigger on. By default KEDA will check each trigger source on every ScaledObject every 30 seconds. + +**Example:** in a queue scenario, KEDA will check the queueLength every `pollingInterval`, and scale the resource up or down accordingly. + +--- + +```yaml + cooldownPeriod: 300 # Optional. Default: 300 seconds +``` + +The period to wait after the last trigger reported active before scaling the resource back to 0. By default it's 5 minutes (300 seconds). + +The `cooldownPeriod` only applies after a trigger occurs; when you first create your `Deployment` (or `StatefulSet`/`CustomResource`), KEDA will immediately scale it to `minReplicaCount`. Additionally, the KEDA `cooldownPeriod` only applies when scaling to 0; scaling from 1 to N replicas is handled by the [Kubernetes Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-cooldowndelay). + +**Example:** wait 5 minutes after the last time KEDA checked the queue and it was empty. (this is obviously dependent on `pollingInterval`) + +--- + +```yaml + idleReplicaCount: 0 # Optional. Must be less than minReplicaCount +``` + +> 💡 **NOTE:** Due to limitations in HPA controller the only supported value for this property is 0, it will not work correctly otherwise. See this [issue](https://github.com/kedacore/keda/issues/2314) for more details. + +If this property is set, KEDA will scale the resource down to this number of replicas. If there's some activity on target triggers KEDA will scale the target resource immediately to `minReplicaCount` and then will be scaling handled by HPA. When there is no activity, the target resource is again scaled down to `idleReplicaCount`. This seting must be less than `minReplicaCount`. + +**Example:** If there's no activity on triggers the target resource is scaled down to `idleReplicaCount` (0), once there is an activity the target resource is immediately scaled to `minReplicaCount` (10) and then up to `maxReplicaCount` (100) as needed. If there's no activity on triggers the resource is again scaled down to `idleReplicaCount` (0). + +--- + +```yaml + minReplicaCount: 1 # Optional. Default: 0 +``` + +Minimum number of replicas KEDA will scale the resource down to. By default it's scale to zero, but you can use it with some other value as well. + +--- + +```yaml + maxReplicaCount: 100 # Optional. Default: 100 +``` + +This setting is passed to the HPA definition that KEDA will create for a given resource and holds the maximum number of replicas of the target resouce. + +--- + +```yaml + fallback: # Optional. Section to specify fallback options + failureThreshold: 3 # Mandatory if fallback section is included + replicas: 6 # Mandatory if fallback section is included +``` + +The `fallback` section is optional. It defines a number of replicas to fallback to if a scaler is in an error state. + +KEDA will keep track of the number of consecutive times each scaler has failed to get metrics from its source. Once that value passes the `failureThreshold`, instead of not propagating a metric to the HPA (the default error behaviour), the scaler will, instead, return a normalised metric using the formula: +``` +target metric value * fallback replicas +``` +Due to the HPA metric being of type `AverageValue` (see below), this will have the effect of the HPA scaling the deployment to the defined number of fallback replicas. + +**Example:** When my instance of prometheus is unavailable 3 consecutive times, KEDA will change the HPA metric such that the deployment will scale to 6 replicas. + +There are a few limitations to using a fallback: + - It is **not** supported by the CPU & memory scalers and will assume that fallback is disabled in those cases. This is because it only supports scalers that present their target as an `AverageValue` metric. + - It is only supported by `ScaledObjects` **not** `ScaledJobs`. + + +--- + +```yaml +advanced: + restoreToOriginalReplicaCount: true/false # Optional. Default: false +``` + +This property specifies whether the target resource (`Deployment`, `StatefulSet`,...) should be scaled back to original replicas count, after the `ScaledObject` is deleted. +Default behavior is to keep the replica count at the same number as it is in the moment of `ScaledObject's` deletion. + +For example a `Deployment` with `3 replicas` is created, then `ScaledObject` is created and the `Deployment` is scaled by KEDA to `10 replicas`. Then `ScaledObject` is deleted: + 1. if `restoreToOriginalReplicaCount = false` (default behavior) then `Deployment` replicas count is `10` + 2. if `restoreToOriginalReplicaCount = true` then `Deployment` replicas count is set back to `3` (the original value) + +--- + +```yaml +advanced: + horizontalPodAutoscalerConfig: # Optional. Section to specify HPA related options + behavior: # Optional. Use to modify HPA's scaling behavior + scaleDown: + stabilizationWindowSeconds: 300 + policies: + - type: Percent + value: 100 + periodSeconds: 15 +``` + +**`horizontalPodAutoscalerConfig:`** + +**`horizontalPodAutoscalerConfig.behavior`:** + +Starting from Kubernetes v1.18 the autoscaling API allows scaling behavior to be configured through the HPA behavior field. This way one can directly affect scaling of 1<->N replicas, which is internally being handled by HPA. KEDA would feed values from this section directly to the HPA's `behavior` field. Please follow [Kubernetes documentation](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-configurable-scaling-behavior) for details. + +**Assumptions:** KEDA must be running on Kubernetes cluster v1.18+, in order to be able to benefit from this setting. + +## Long-running executions + +One important consideration to make is how this pattern can work with long running executions. Imagine a deployment triggers on a RabbitMQ queue message. Each message takes 3 hours to process. It's possible that if many queue messages arrive, KEDA will help drive scaling out to many replicas - let's say 4. Now the HPA makes a decision to scale down from 4 replicas to 2. There is no way to control which of the 2 replicas get terminated to scale down. That means the HPA may attempt to terminate a replica that is 2.9 hours into processing a 3 hour queue message. + +There are two main ways to handle this scenario. + +### Leverage the container lifecycle + +Kubernetes provides a few [lifecycle hooks](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/) that can be leveraged to delay termination. Imagine a replica is scheduled for termination and is 2.9 hours into processing a 3 hour message. Kubernetes will send a [`SIGTERM`](https://www.gnu.org/software/libc/manual/html_node/Termination-Signals.html) to signal the intent to terminate. Rather than immediately terminating, a deployment can delay termination until processing the current batch of messages has completed. Kubernetes will wait for a `SIGTERM` response or the `terminationGracePeriodSeconds` before killing the replica. + +> 💡 **NOTE:** There are other ways to delay termination, including the [`preStop` Hook](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks). + +Using this method can preserve a replica and enable long-running executions. However, one downside of this approach is while delaying termination, the pod phase will remain in the `Terminating` state. That means a pod that is delaying termination for a very long duration may show `Terminating` during that entire period of delay. + +### Run as jobs + +The other alternative to handling long running executions is by running the event driven code in Kubernetes Jobs instead of Deployments or Custom Resources. This approach is discussed [in the next section](../scaling-jobs). diff --git a/content/docs/2.7/concepts/scaling-jobs.md b/content/docs/2.7/concepts/scaling-jobs.md new file mode 100644 index 00000000000..a2347a6a21a --- /dev/null +++ b/content/docs/2.7/concepts/scaling-jobs.md @@ -0,0 +1,267 @@ ++++ +title = "Scaling Jobs" +weight = 300 ++++ + + +## Overview + +As an alternate to [scaling event-driven code as deployments](../scaling-deployments) you can also run and scale your code as Kubernetes Jobs. The primary reason to consider this option is to handle processing long running executions. Rather than processing multiple events within a deployment, for each detected event a single Kubernetes Job is scheduled. That job will initialize, pull a single event from the message source, and process to completion and terminate. + +For example, if you wanted to use KEDA to run a job for each message that lands on a RabbitMQ queue, the flow may be: + +1. When no messages are awaiting processing, no jobs are created. +1. When a message arrives on the queue, KEDA creates a job. +1. When the job starts running, it pulls *a single* message and processes it to completion. +1. As additional messages arrive, additional jobs are created. Each job processes a single message to completion. +1. Periodically remove completed/failed job by the `SuccessfulJobsHistoryLimit` and `FailedJobsHistoryLimit.` + +## ScaledJob spec + +This specification describes the `ScaledJob` custom resource definition which is used to define how KEDA should scale your application and what the triggers are. + +[`scaledjob_types.go`](https://github.com/kedacore/keda/blob/main/api/v1alpha1/scaledjob_types.go) + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledJob +metadata: + name: {scaled-job-name} +spec: + jobTargetRef: + parallelism: 1 # [max number of desired pods](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#controlling-parallelism) + completions: 1 # [desired number of successfully finished pods](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#controlling-parallelism) + activeDeadlineSeconds: 600 # Specifies the duration in seconds relative to the startTime that the job may be active before the system tries to terminate it; value must be positive integer + backoffLimit: 6 # Specifies the number of retries before marking this job failed. Defaults to 6 + template: + # describes the [job template](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/) + pollingInterval: 30 # Optional. Default: 30 seconds + successfulJobsHistoryLimit: 5 # Optional. Default: 100. How many completed jobs should be kept. + failedJobsHistoryLimit: 5 # Optional. Default: 100. How many failed jobs should be kept. + envSourceContainerName: {container-name} # Optional. Default: .spec.JobTargetRef.template.spec.containers[0] + maxReplicaCount: 100 # Optional. Default: 100 + rolloutStrategy: gradual # Optional. Default: default. Which Rollout Strategy KEDA will use. + scalingStrategy: + strategy: "custom" # Optional. Default: default. Which Scaling Strategy to use. + customScalingQueueLengthDeduction: 1 # Optional. A parameter to optimize custom ScalingStrategy. + customScalingRunningJobPercentage: "0.5" # Optional. A parameter to optimize custom ScalingStrategy. + pendingPodConditions: # Optional. A parameter to calculate pending job count per the specified pod conditions + - "Ready" + - "PodScheduled" + - "AnyOtherCustomPodCondition" + multipleScalersCalculation : "max" # Optional. Default: max. Specifies how to calculate the target metrics when multiple scalers are defined. + triggers: + # {list of triggers to create jobs} +``` + +You can find all supported triggers [here](../scalers). + +## Details + +```yaml + jobTargetRef: + parallelism: 1 # Max number of desired instances ([docs](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#controlling-parallelism)) + completions: 1 # Desired number of successfully finished instances ([docs](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#controlling-parallelism)) + activeDeadlineSeconds: 600 # Specifies the duration in seconds relative to the startTime that the job may be active before the system tries to terminate it; value must be positive integer + backoffLimit: 6 # Specifies the number of retries before marking this job failed. Defaults to 6 +``` + +--- + +```yaml + pollingInterval: 30 # Optional. Default: 30 seconds +``` + +This is the interval to check each trigger on. By default, KEDA will check each trigger source on every ScaledJob every 30 seconds. + +--- + +```yaml + successfulJobsHistoryLimit: 5 # Optional. Default: 100. How many completed jobs should be kept. + failedJobsHistoryLimit: 5 # Optional. Default: 100. How many failed jobs should be kept. +``` + +The `successfulJobsHistoryLimit` and `failedJobsHistoryLimit` fields are optional. These fields specify how many completed and failed jobs should be kept. By default, they are set to 100. + +This concept is similar to [Jobs History Limits](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#jobs-history-limits) allowing you to learn what the outcomes of your jobs are. + +The actual number of jobs could exceed the limit in a short time. However, it is going to resolve in the cleanup period. Currently, the cleanup period is the same as the Polling interval. + +--- + + +```yaml + envSourceContainerName: {container-name} # Optional. Default: .spec.JobTargetRef.template.spec.containers[0] +``` + +This optional property specifies the name of container in the Job, from which KEDA should try to get environment properties holding secrets etc. If it is not defined it, KEDA will try to get environment properties from the first Container, ie. from `.spec.JobTargetRef.template.spec.containers[0]`. + + +--- + +```yaml + maxReplicaCount: 100 # Optional. Default: 100 +``` + +The max number of pods that is created within a single polling period. If there are running jobs, the number of running jobs will be deducted. This table is an example of the scaling logic. + +| Queue Length | Max Replica Count | Target Average Value | Running Job Count | Number of the Scale | +| ------- | ------ | ------- | ------ | ----- | +| 10 | 3 | 1 | 0 | 3 | +| 10 | 3 | 2 | 0 | 3 | +| 10 | 3 | 1 | 1 | 2 | +| 10 | 100 | 1 | 0 | 10 | +| 4 | 3 | 5 | 0 | 1 | + +* **Queue Length:** The number of items in the queue. +* **Target Average Value:** The number of messages that will be consumed on a job. It is defined on the scaler side. e.g. `queueLength` on `Azure Storage Queue` scaler. +* **Running Job Count:** How many jobs are running. +* **Number of the Scale:** The number of the job that is created. + +--- + +```yaml + rolloutStrategy: default # Optional. Default: default. Which Rollout Strategy KEDA will use. +``` + +This optional property specifies the rollout strategy KEDA will use while updating an existing ScaledJob. +Possible values are `default` or `gradual`. \ +When using the `default` rolloutStrategy, KEDA will terminate existing Jobs whenever a ScaledJob is being updated. Then, it will recreate those Jobs with the latest specs. \ +On the `gradual` rolloutStartegy, whenever a ScaledJob is being updated, KEDA will not delete existing Jobs. Only new Jobs will be created with the latest specs. + + +--- + +```yaml +scalingStrategy: + strategy: "default" # Optional. Default: default. Which Scaling Strategy to use. +``` + +Select a Scaling Strategy. Possible values are `default`, `custom`, or `accurate`. The default value is `default`. + +> 💡 **NOTE:** +> +>`maxScale` is not the running Job count. It is measured as follows: + >```go + >maxValue = min(scaledJob.MaxReplicaCount(), divideWithCeil(queueLength, targetAverageValue)) + >``` + >That means it will use the value of `queueLength` divided by `targetAvarageValue` unless it is exceeding the `MaxReplicaCount`. +> +>`RunningJobCount` represents the number of jobs that are currently running or have not finished yet. +> +>It is measured as follows: +>```go +>if !e.isJobFinished(&job) { +> runningJobs++ +>} +>``` +>`PendingJobCount` provides an indication of the amount of jobs that are in pending state. Pending jobs can be calculated in two ways: +> - Default behavior - Job that have not finished yet **and** the underlying pod is either not running or has not been completed yet +> - Setting `pendingPodConditions` - Job that has not finished yet **and** all specified pod conditions of the underlying pod mark as `true` by kubernetes. +> +>It is measured as follows: +>```go +>if !e.isJobFinished(&job) { +> if len(scaledJob.Spec.ScalingStrategy.PendingPodConditions) > 0 { +> if !e.areAllPendingPodConditionsFulfilled(&job, scaledJob.Spec.ScalingStrategy.PendingPodConditions) { +> pendingJobs++ +> } +> } else { +> if !e.isAnyPodRunningOrCompleted(&job) { +> pendingJobs++ +> } +> } +>} +>``` + +**default** +This logic is the same as Job for V1. The number of the scale will be calculated as follows. + +_The number of the scale_ + +```go +maxScale - runningJobCount +``` + +**custom** +You can customize the default scale logic. You need to configure the following parameters. If you don't configure it, then the strategy will be `default.` + +```yaml +customScalingQueueLengthDeduction: 1 # Optional. A parameter to optimize custom ScalingStrategy. +customScalingRunningJobPercentage: "0.5" # Optional. A parameter to optimize custom ScalingStrategy. +``` + +_The number of the scale_ + +```go +min(maxScale-int64(*s.CustomScalingQueueLengthDeduction)-int64(float64(runningJobCount)*(*s.CustomScalingRunningJobPercentage)), maxReplicaCount) +``` + +**accurate** +If the scaler returns `queueLength` (number of items in the queue) that does not include the number of locked messages, this strategy is recommended. `Azure Storage Queue` is one example. You can use this strategy if you delete a message once your app consumes it. + +```go +if (maxScale + runningJobCount) > maxReplicaCount { + return maxReplicaCount - runningJobCount + } + return maxScale - pendingJobCount +``` +For more details, you can refer to [this PR](https://github.com/kedacore/keda/pull/1227). + +--- + +```yaml +scalingStrategy: + multipleScalersCalculation : "max" # Optional. Default: max. Specifies how to calculate the target metrics (`queueLength` and `maxValue`) when multiple scalers are defined. +``` +Select a behavior if you have multiple triggers. Possible values are `max`, `min`, `avg`, or `sum`. The default value is `max`. + +* **max:** - Use metrics from the scaler that has the max number of `queueLength`. (default) +* **min:** - Use metrics from the scaler that has the min number of `queueLength`. +* **avg:** - Sum up all the active scalers metrics and divide by the number of active scalers. +* **sum:** - Sum up all the active scalers metrics. + +# Sample + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: rabbitmq-consumer +data: + RabbitMqHost: +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledJob +metadata: + name: rabbitmq-consumer + namespace: default +spec: + jobTargetRef: + template: + spec: + containers: + - name: rabbitmq-client + image: tsuyoshiushio/rabbitmq-client:dev3 + imagePullPolicy: Always + command: ["receive", "amqp://user:PASSWORD@rabbitmq.default.svc.cluster.local:5672", "job"] + envFrom: + - secretRef: + name: rabbitmq-consumer + restartPolicy: Never + backoffLimit: 4 + pollingInterval: 10 # Optional. Default: 30 seconds + maxReplicaCount: 30 # Optional. Default: 100 + successfulJobsHistoryLimit: 3 # Optional. Default: 100. How many completed jobs should be kept. + failedJobsHistoryLimit: 2 # Optional. Default: 100. How many failed jobs should be kept. + scalingStrategy: + strategy: "custom" # Optional. Default: default. Which Scaling Strategy to use. + customScalingQueueLengthDeduction: 1 # Optional. A parameter to optimize custom ScalingStrategy. + customScalingRunningJobPercentage: "0.5" # Optional. A parameter to optimize custom ScalingStrategy. + triggers: + - type: rabbitmq + metadata: + queueName: hello + host: RabbitMqHost + queueLength : '5' +``` diff --git a/content/docs/2.7/concepts/troubleshooting.md b/content/docs/2.7/concepts/troubleshooting.md new file mode 100644 index 00000000000..68b19bf15b9 --- /dev/null +++ b/content/docs/2.7/concepts/troubleshooting.md @@ -0,0 +1,24 @@ ++++ +title = "Troubleshooting" +weight = 600 ++++ + +## KEDA logging and telemetry + +The first place to look if something isn't behaving correctly is the logs generated from KEDA. After deploying you should have a pod with two containers running within the namespace (by default: `keda`). + +You can view the KEDA operator pod via kubectl: + +```sh +kubectl get pods -n keda +``` + +You can view the logs for the keda operator container with the following: + +```sh +kubectl logs -n keda {keda-pod-name} -c keda-operator +``` + +## Reporting issues + +If you are having issues or hitting a potential bug, please file an issue [in the KEDA GitHub repo](https://github.com/kedacore/keda/issues/new/choose) with details, logs, and steps to reproduce the behavior. diff --git a/content/docs/2.7/deploy.md b/content/docs/2.7/deploy.md new file mode 100644 index 00000000000..f72a9d587bf --- /dev/null +++ b/content/docs/2.7/deploy.md @@ -0,0 +1,127 @@ ++++ +title = "Deploying KEDA" ++++ + +We provide a few approaches to deploy KEDA runtime in your Kubernetes clusters: + +- [Helm charts](#helm) +- [Operator Hub](#operatorhub) +- [YAML declarations](#yaml) + +> 💡 **NOTE:** KEDA requires Kubernetes cluster version 1.16 and higher + +Don't see what you need? Feel free to [create an issue](https://github.com/kedacore/keda/issues/new) on our GitHub repo. + +## Deploying with Helm {#helm} + +### Install + +Deploying KEDA with Helm is very simple: + +1. Add Helm repo + + ```sh + helm repo add kedacore https://kedacore.github.io/charts + ``` + +2. Update Helm repo + + ```sh + helm repo update + ``` + +3. Install `keda` Helm chart + + **Helm 3** + + ```sh + kubectl create namespace keda + helm install keda kedacore/keda --namespace keda + ``` + +> 💡 **NOTE:** Are you upgrading to v2.2.1 or above? Make sure to read [our troubleshooting guide](https://keda.sh/docs/latest/troubleshooting/) to fix potential CRD issues. + +### Uninstall + +If you want to remove KEDA from a cluster you can run one of the following: + +```sh +helm uninstall keda -n keda +``` + +## Deploying with Operator Hub {#operatorhub} + +### Install + +1. On Operator Hub Marketplace locate and install KEDA operator to namespace `keda` +2. Create `KedaController` resource named `keda` in namespace `keda` +![Operator Hub installation](https://raw.githubusercontent.com/kedacore/keda-olm-operator/main/images/keda-olm-install.gif) +> 💡 **NOTE:** Further information on Operator Hub installation method can be found in the following [repository](https://github.com/kedacore/keda-olm-operator). + +### Uninstall + +Locate installed KEDA Operator in `keda` namespace, then remove created `KedaController` resource and uninstall KEDA operator. + +## Deploying using the deployment YAML files {#yaml} + +### Install + +If you want to try KEDA on [Minikube](https://minikube.sigs.k8s.io) or a different Kubernetes deployment without using Helm you can still deploy it with `kubectl`. + +- We provide sample YAML declaration which includes our CRDs and all other resources in a file which is available on the [GitHub releases](https://github.com/kedacore/keda/releases) page. +Run the following command (if needed, replace the version, in this case `2.7.0`, with the one you are using): + +```sh +kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.7.0/keda-2.7.0.yaml +``` + +- Alternatively you can download the file and deploy it from the local path: +```sh +kubectl apply -f keda-2..0.yaml +``` + +- You can also find the same YAML declarations in our `/config` directory on our [GitHub repo](https://github.com/kedacore/keda) if you prefer to clone it. + +```sh +git clone https://github.com/kedacore/keda && cd keda + +VERSION=2.7.0 make deploy +``` + +### Uninstall + +- In case of installing from released YAML file just run the following command (if needed, replace the version, in this case `2.7.0`, with the one you are using): + +```sh +kubectl delete -f https://github.com/kedacore/keda/releases/download/v2.7.0/keda-2.7.0.yaml +``` + +- If you have downloaded the file locally, you can run: + +```sh +kubectl delete -f keda-2.7.0.yaml +``` + +- You would need to run these commands from within the directory of the cloned [GitHub repo](https://github.com/kedacore/keda): + +```sh +VERSION=2.7.0 make undeploy +``` + +## Deploying KEDA on MicroK8s {#microk8s} + +### Install + +If you want to try KEDA v2 on [MicroK8s](https://microk8s.io/) from `1.20` channel, KEDA is included into MicroK8s addons. + +```sh +microk8s enable keda +``` + +### Uninstall + +To uninstall KEDA in MicroK8s, simply disable the addon as shown below. + +```sh +microk8s disable keda +``` \ No newline at end of file diff --git a/content/docs/2.7/faq.md b/content/docs/2.7/faq.md new file mode 100644 index 00000000000..d012d676750 --- /dev/null +++ b/content/docs/2.7/faq.md @@ -0,0 +1,5 @@ ++++ +title = "FAQ" ++++ + +{{< faq20 >}} diff --git a/content/docs/2.7/migration.md b/content/docs/2.7/migration.md new file mode 100644 index 00000000000..5ee8a6455d1 --- /dev/null +++ b/content/docs/2.7/migration.md @@ -0,0 +1,200 @@ ++++ +title = "Migration Guide" ++++ + +## Migrating from KEDA v1 to v2 + +Please note that you **can not** run both KEDA v1 and v2 on the same Kubernetes cluster. You need to [uninstall](../../1.5/deploy) KEDA v1 first, in order to [install](../deploy) and use KEDA v2. + +> 💡 **NOTE:** When uninstalling KEDA v1 make sure v1 CRDs are uninstalled from the cluster as well. + +KEDA v2 is using a new API namespace for its Custom Resources Definitions (CRD): `keda.sh` instead of `keda.k8s.io` and introduces a new Custom Resource for scaling of Jobs. See full details on KEDA Custom Resources [here](../concepts/#custom-resources-crd). + +Here's an overview of what's changed: +- [Scaling of Deployments](#scaling-of-deployments) +- [Scaling of Jobs](#scaling-of-jobs) +- [Improved flexibility & usability of trigger metadata](#improved-flexibility--usability-of-trigger-metadata) +- [Scalers](#scalers) +- [TriggerAuthentication](#triggerauthentication) + + +### Scaling of Deployments +In order to scale `Deployments` with KEDA v2, you need to do only a few modifications to existing v1 `ScaledObjects` definitions, so they comply with v2: +- Change the value of `apiVersion` property from `keda.k8s.io/v1alpha1` to `keda.sh/v1alpha1` +- Rename property `spec.scaleTargetRef.deploymentName` to `spec.scaleTargetRef.name` +- Rename property `spec.scaleTargetRef.containerName` to `spec.scaleTargetRef.envSourceContainerName` +- Label `deploymentName` (in `metadata.labels.`) is no longer needed to be specified on v2 ScaledObject (it was mandatory on older versions of v1) + +Please see the examples below or refer to the full [v2 ScaledObject Specification](../concepts/scaling-deployments/#scaledobject-spec) + + **Example of v1 ScaledObject** +```yaml +apiVersion: keda.k8s.io/v1alpha1 +kind: ScaledObject +metadata: + name: {scaled-object-name} + labels: + deploymentName: {deployment-name} +spec: + scaleTargetRef: + deploymentName: {deployment-name} + containerName: {container-name} + pollingInterval: 30 + cooldownPeriod: 300 + minReplicaCount: 0 + maxReplicaCount: 100 + triggers: + # {list of triggers to activate the deployment} +``` + +**Example of v2 ScaledObject** + +```yaml +apiVersion: keda.sh/v1alpha1 # <--- Property value was changed +kind: ScaledObject +metadata: # <--- labels.deploymentName is not needed + name: {scaled-object-name} +spec: + scaleTargetRef: + name: {deployment-name} # <--- Property name was changed + envSourceContainerName: {container-name} # <--- Property name was changed + pollingInterval: 30 + cooldownPeriod: 300 + minReplicaCount: 0 + maxReplicaCount: 100 + triggers: + # {list of triggers to activate the deployment} +``` + +### Scaling of Jobs + +In order to scale `Jobs` with KEDA v2, you need to do only a few modifications to existing v1 `ScaledObjects` definitions, so they comply with v2: +- Change the value of `apiVersion` property from `keda.k8s.io/v1alpha1` to `keda.sh/v1alpha1` +- Change the value of `kind` property from `ScaledObject` to `ScaledJob` +- Remove property `spec.scaleType` +- Remove properties `spec.cooldownPeriod` and `spec.minReplicaCount` + +You can configure `successfulJobsHistoryLimit` and `failedJobsHistoryLimit`. They will remove the old job histories automatically. + +Please see the examples below or refer to the full [v2 ScaledJob Specification](../concepts/scaling-jobs/#scaledjob-spec) + +**Example of v1 ScaledObject for Jobs scaling** +```yaml +apiVersion: keda.k8s.io/v1alpha1 +kind: ScaledObject +metadata: + name: {scaled-object-name} +spec: + scaleType: job + jobTargetRef: + parallelism: 1 + completions: 1 + activeDeadlineSeconds: 600 + backoffLimit: 6 + template: + # {job template} + pollingInterval: 30 + cooldownPeriod: 300 + minReplicaCount: 0 + maxReplicaCount: 100 + triggers: + # {list of triggers to create jobs} +``` + +**Example of v2 ScaledJob** + +```yaml +apiVersion: keda.sh/v1alpha1 # <--- Property value was changed +kind: ScaledJob # <--- Property value was changed +metadata: + name: {scaled-job-name} +spec: # <--- spec.scaleType is not needed + jobTargetRef: + parallelism: 1 + completions: 1 + activeDeadlineSeconds: 600 + backoffLimit: 6 + template: + # {job template} + pollingInterval: 30 # <--- spec.cooldownPeriod and spec.minReplicaCount are not needed + successfulJobsHistoryLimit: 5 # <--- property is added + failedJobsHistoryLimit: 5 # <--- Property is added + maxReplicaCount: 100 + triggers: + # {list of triggers to create jobs} +``` + +### Improved flexibility & usability of trigger metadata + +We've introduced more options to configure trigger metadata to give users more flexibility. + +> 💡 **NOTE:** Changes only apply to trigger metadata and don't impact usage of `TriggerAuthentication` + +Here's an overview: + +| Scaler | 1.x | 2.0 | +|--------|--------|--------| +| `azure-blob` | `connection` (**Default**: `AzureWebJobsStorage`) | `connectionFromEnv` | +| `azure-monitor` | `activeDirectoryClientId`
`activeDirectoryClientPassword` | `activeDirectoryClientId`
`activeDirectoryClientIdFromEnv`
`activeDirectoryClientPasswordFromEnv` | +| `azure-queue` | `connection` (**Default**: AzureWebJobsStorage) | `connectionFromEnv` | +| `azure-servicebus` | `connection` | `connectionFromEnv` | +| `azure-eventhub` | `storageConnection` (**Default**: `AzureWebJobsStorage`)
`connection` (**Default**: `EventHub`) | `storageConnectionFromEnv`
`connectionFromEnv` | +| `aws-cloudwatch` | `awsAccessKeyID` (**Default**: `AWS_ACCESS_KEY_ID`)
`awsSecretAccessKey` (**Default**: `AWS_SECRET_ACCESS_KEY`) | `awsAccessKeyID`
`awsAccessKeyIDFromEnv`
`awsSecretAccessKeyFromEnv` | +| `aws-kinesis-stream` | `awsAccessKeyID` (**Default**: `AWS_ACCESS_KEY_ID`)
`awsSecretAccessKey` (**Default**: `AWS_SECRET_ACCESS_KEY`) | `awsAccessKeyID`
`awsAccessKeyIDFromEnv`
`awsSecretAccessKeyFromEnv` | +| `aws-sqs-queue` | `awsAccessKeyID` (**Default**: `AWS_ACCESS_KEY_ID`)
`awsSecretAccessKey` (**Default**: `AWS_SECRET_ACCESS_KEY`) | `awsAccessKeyID`
`awsAccessKeyIDFromEnv`
`awsSecretAccessKeyFromEnv` | +| `kafka` | _(none)_ | _(none)_ +| `rabbitmq` | `apiHost`
`host` | ~~`apiHost`~~
`host`
`hostFromEnv` | +| `prometheus` | _(none)_ | _(none)_ | +| `cron` | _(none)_ | _(none)_ | +| `redis` | `address`
`host`
`port`
`password` | `address`
`addressFromEnv`
`host`
`hostFromEnv`
~~`port`~~
`passwordFromEnv` +| `redis-streams` | `address`
`host`
`port`
`password` | `address`
`addressFromEnv`
`host`
`hostFromEnv`
~~`port`~~
`passwordFromEnv` +| `gcp-pubsub` | `credentials` | `credentialsFromEnv` | +| `external` | _(any matching value)_ | _(any matching value with `FromEnv` suffix)_ +| `liiklus` | _(none)_ | _(none)_ | +| `stan` | _(none)_ | _(none)_ | +| `huawei-cloudeye` | | _(none)_ | _(none)_ | +| `postgresql` | `connection`
`password` | `connectionFromEnv`
`passwordFromEnv` | +| `mysql` | `connectionString`
`password` | `connectionStringFromEnv`
`passwordFromEnv` | + +### Scalers + +**Azure Service Bus** + - `queueLength` was renamed to `messageCount` + +**Kafka** + - `authMode` property was replaced with `sasl` and `tls` properties. Please refer [documentation](../scalers/apache-kafka/#authentication-parameters) for Kafka Authentication Parameters details. + +**RabbitMQ** + +In KEDA 2.0 the RabbitMQ scaler has only `host` parameter, and the protocol for communication can be specified by +`protocol` (http or amqp). The default value is `amqp`. The behavior changes only for scalers that were using HTTP +protocol. + +Example of RabbitMQ trigger before 2.0: +```yaml +triggers: +- type: rabbitmq + metadata: + queueLength: '20' + queueName: testqueue + includeUnacked: 'true' + apiHost: 'https://guest:password@localhost:443/vhostname' +``` + +The same trigger in 2.0: +```yaml +triggers: +- type: rabbitmq + metadata: + queueLength: '20' + queueName: testqueue + protocol: 'http' + host: 'https://guest:password@localhost:443/vhostname' +``` + +### TriggerAuthentication +In order to use Authentication via `TriggerAuthentication` with KEDA v2, you need to change: +- Change the value of `apiVersion` property from `keda.k8s.io/v1alpha1` to `keda.sh/v1alpha1` + +For more details please refer to the full +[v2 TriggerAuthentication Specification](../concepts/authentication/#re-use-credentials-and-delegate-auth-with-triggerauthentication) diff --git a/content/docs/2.7/operate/_index.md b/content/docs/2.7/operate/_index.md new file mode 100644 index 00000000000..631386165a1 --- /dev/null +++ b/content/docs/2.7/operate/_index.md @@ -0,0 +1,12 @@ ++++ +title = "Operate" +description = "Guidance & requirements for operating KEDA" +weight = 1 ++++ + +We provide guidance & requirements around various areas to operate KEDA: + +- Cluster ([link](./cluster)) +- Integrate with Prometheus ([link](./prometheus)) +- Kubernetes Events ([link](./events)) +- KEDA Metrics Server ([link](./metrics-server)) \ No newline at end of file diff --git a/content/docs/2.7/operate/cluster.md b/content/docs/2.7/operate/cluster.md new file mode 100644 index 00000000000..90da96cafef --- /dev/null +++ b/content/docs/2.7/operate/cluster.md @@ -0,0 +1,87 @@ ++++ +title = "Cluster" +description = "Guidance & requirements for running KEDA in your cluster" +weight = 100 ++++ + +## Cluster capacity requirements + +The KEDA runtime require the following resources in a production-ready setup: + +| Deployment | CPU | Memory | +| -------------- | ----------------------- | ----------------------------- | +| Metrics Server | Limit: 1, Request: 100m | Limit: 1000Mi, Request: 100Mi | +| Operator | Limit: 1, Request: 100m | Limit: 1000Mi, Request: 100Mi | + +These are used by default when deploying through YAML. + +> 💡 For more info on CPU and Memory resource units and their meaning, see [this](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#resource-units-in-kubernetes) link. + +## Firewall requirements + +KEDA requires to be accessible inside the cluster to be able to autoscale. + +Here is an overview of the required ports that need to be accessible for KEDA to work: + + +| Port | Why? | Remarks | +| ------ | -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `443` | Used by Kubernetes API server to get metrics | Required for all platforms because it uses Control Plane → port 443 on the Service IP range communication.

This is not applicable for Google Cloud. | +| `6443` | Used by Kubernetes API server to get metrics | Only required for Google Cloud because it uses Control Plane → port 6443 on the Pod IP range for communication | + + +## High Availability + +KEDA does not provide support for high-availability due to upstream limitations. + +Here is an overview of all KEDA deployments and the supported replicas: + +| Deployment | Support Replicas | Reasoning | +| -------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------- | +| Metrics Server | 1 | Limitation in [k8s custom metrics server](https://github.com/kubernetes-sigs/custom-metrics-apiserver/issues/70) | +| Operator | 2 | While you can run more replicas of our operator, only one operator instance will be active. The rest will be standing by, which may reduce downtime during a failure. Multiple replicas will not improve the performance of KEDA, it could only reduce a downtime during a failover.| + +## HTTP Timeouts + +Some scalers issue HTTP requests to external servers (i.e. cloud services). Each applicable scaler uses its own dedicated HTTP client with its own connection pool, and by default each client is set to time out any HTTP request after 3 seconds. + +You can override this default by setting the `KEDA_HTTP_DEFAULT_TIMEOUT` environment variable to your desired timeout in milliseconds. For example, on Linux/Mac/Windows WSL2 operating systems, you'd use this command to set to 1 second: + +```shell +export KEDA_HTTP_DEFAULT_TIMEOUT=1000 +``` + +And on Windows Powershell, you'd use this command: + +```shell +$env:KEDA_HTTP_DEFAULT_TIMEOUT=1000 +``` + +All applicable scalers will use this timeout. Setting a per-scaler timeout is currently unsupported. + +## Kubernetes Client Parameters + +The Kubernetes client config used within KEDA Metrics Adapter can be adjusted by passing the following command-line flags to the binary: + +| Adapter Flag | Client Config Setting | Default Value | Description | +| -------------- | ----------------------- | ------------- | -------------------------------------------------------------- | +| kube-api-qps | cfg.QPS | 20.0 | Set the QPS rate for throttling requests sent to the apiserver | +| kube-api-burst | cfg.Burst | 30 | Set the burst for throttling requests sent to the apiserver | + +## Configure `MaxConcurrentReconciles` for Controllers + +To implement internal controllers KEDA uses [controller-runtime project](https://github.com/kubernetes-sigs/controller-runtime), that enables configuration of [MaxConcurrentReconciles property](https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller#Options), ie. the maximum number of concurrent reconciles which can be run for a controller. + +KEDA Operator exposes properties for specifying `MaxConcurrentReconciles` for following controllers/reconcilers: +- `ScaledObjectReconciler` - responsible for watching and managing `ScaledObjects`, ie. validates input trigger specification, starts scaling logic and manages dependent HPA. +- `ScaledJobReconciler` - responsible for watching and managing `ScaledJobs` and dependent Kubernetes Jobs + +KEDA Metrics Server exposes property for specifying `MaxConcurrentReconciles` for `MetricsScaledObjectReconciler`, that manages Metrics Names exposes by KEDA and which are being consumed by Kubernetes server and HPA controller. + +To modify this properties you can set environment variables on both KEDA Operator and Metrics Server Deployments: + +| Environment variable name | Deployment | Default Value | Affected reconciler | +| ------------------------------------- | -------------- | ------------- | -------------------------------------------------------------- | +| KEDA_SCALEDOBJECT_CTRL_MAX_RECONCILES | Operator | 5 | ScaledObjectReconciler | +| KEDA_SCALEDJOB_CTRL_MAX_RECONCILES | Operator | 1 | ScaledJobReconciler | +| KEDA_METRICS_CTRL_MAX_RECONCILES | Metrics Server | 1 | MetricsScaledObjectReconciler | diff --git a/content/docs/2.7/operate/events.md b/content/docs/2.7/operate/events.md new file mode 100644 index 00000000000..adb16b9e47f --- /dev/null +++ b/content/docs/2.7/operate/events.md @@ -0,0 +1,30 @@ ++++ +title = "Events" +description = "Kubernetes Events emitted by KEDA" +weight = 100 ++++ + +## Kubernetes Events emitted by KEDA + +KEDA emits the following [Kubernetes Events](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#event-v1-core): + +| Event | Type | Description | +|-------------------------------------|-----------|-----------------------------------------------------------------------------------------------------------------------------| +| `ScaledObjectReady` | `Normal` | On the first time a ScaledObject is ready, or if the previous ready condition status of the object was `Unknown` or `False` | +| `ScaledJobReady` | `Normal` | On the first time a ScaledJob is ready, or if the previous ready condition status of the object was `Unknown` or `False` | +| `ScaledObjectCheckFailed` | `Warning` | If the check validation for a ScaledObject fails | +| `ScaledJobCheckFailed` | `Warning` | If the check validation for a ScaledJob fails | +| `ScaledObjectDeleted` | `Normal` | When a ScaledObject is deleted and removed from KEDA watch | +| `ScaledJobDeleted` | `Normal` | When a ScaledJob is deleted and removed from KEDA watch | +| `KEDAScalersStarted` | `Normal` | When Scalers watch loop have started for a ScaledObject or ScaledJob | +| `KEDAScalersStopped` | `Normal` | When Scalers watch loop have stopped for a ScaledObject or a ScaledJob | +| `KEDAScalerFailed` | `Warning` | When a Scaler fails to create or check its event source | +| `KEDAScaleTargetActivated` | `Normal` | When the scale target (Deployment, StatefulSet, etc) of a ScaledObject is scaled to 1 | +| `KEDAScaleTargetDeactivated` | `Normal` | When the scale target (Deployment, StatefulSet, etc) of a ScaledObject is scaled to 0 | +| `KEDAScaleTargetActivationFailed` | `Warning` | When KEDA fails to scale the scale target of a ScaledObject to 1 | +| `KEDAScaleTargetDeactivationFailed` | `Warning` | When KEDA fails to scale the scale target of a ScaledObject to 0 | +| `KEDAJobsCreated` | `Normal` | When KEDA creates jobs for a ScaledJob | +| `TriggerAuthenticationAdded` | `Normal` | When a new TriggerAuthentication is added | +| `TriggerAuthenticationDeleted` | `Normal` | When a TriggerAuthentication is deleted | +| `ClusterTriggerAuthenticationAdded` | `Normal` | When a new ClusterTriggerAuthentication is added | +| `ClusterTriggerAuthenticationDeleted`| `Normal` | When a ClusterTriggerAuthentication is deleted | diff --git a/content/docs/2.7/operate/metrics-server.md b/content/docs/2.7/operate/metrics-server.md new file mode 100644 index 00000000000..759c874ac9b --- /dev/null +++ b/content/docs/2.7/operate/metrics-server.md @@ -0,0 +1,97 @@ ++++ +title = "KEDA Metrics Server" +description = "Details on KEDA Metrics Server" +weight = 100 ++++ + +## Querying metrics exposed by KEDA Metrics Server + +The metrics exposed by KEDA Metrics Server can be queried directly using `kubectl`: +```bash +kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1" +``` + +This will return a json with the list of metrics exposed by KEDA: +```json +{ + "kind": "APIResourceList", + "apiVersion": "v1", + "groupVersion": "external.metrics.k8s.io/v1beta1", + "resources": [ + { + "name": "s0-rabbitmq-queueName", + "singularName": "", + "namespaced": true, + "kind": "ExternalMetricValueList", + "verbs": [ + "get" + ] + }, + { + "name": "s1-rabbitmq-queueName2", + .... + } + ] +} +``` + +You can also query for the value of a specifyc metric using `kubectl`: +```bash +kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/YOUR_NAMESPACE/YOUR_METRIC_NAME" +``` + +At this point, you should take in consideration that KEDA metrics are namespaced, this means that you have to specify the namespace where the `ScaledObject` is placed inside. + +For example, if you want to get the value of the metric named `s1-rabbitmq-queueName2` in namespace `sample-ns`, the query will be like this: +```bash +kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/sample-ns/s1-rabbitmq-queueName2" +``` + +And it will show a json like this: + +```json +{ + "kind": "ExternalMetricValueList", + "apiVersion": "external.metrics.k8s.io/v1beta1", + "metadata": {}, + "items": [ + { + "metricName": "s1-rabbitmq-queueName2", + "metricLabels": null, + "timestamp": "2021-10-20T10:48:17Z", + "value": "0" + } + ] +} +``` + +> **Note:** There are 2 exceptions in querying metrics and those are `cpu` and `memory` scalers. When KEDA creates the HPA object, it uses standard `cpu` and `memory` metrics from the Kubernetes Metrics Server. If you want to query these 2 specific values, you should do it using `/apis/metrics.k8s.io/v1beta1` instead of `/apis/external.metrics.k8s.io/v1beta1`. + +## How to get metric names from ScaledObject + +During its work, KEDA updates each ScaledObject with some relevant information which it needs to work. Part of that information is metric names generated from the triggers inside the own ScaledObject. + +You can recover the metric names from a ScaledObject using `kubectl`: +```bash + kubectl get scaledobject SCALEDOBJECT_NAME -n NAMESPACE -o jsonpath={.status.externalMetricNames} +``` + +## How to get metric when multiple ScaledObjects have the same metric name + +KEDA will try to select the proper `ScaledObject` for your metric and there should only be one. In case of having multiple `ScaledObject`s in the same namespace with the same metric name, an error like this will be thrown: + +``` +Error from server: exactly one ScaledObject should match label +``` + +In this case, you should add in the query string the `labelSelector` to match the proper `ScaledObject` (in url format). The needed selector is `scaledobject.keda.sh/name: {ScaledObjectName}`. You can achieve this as following: + +```bash + kubectl get scaledobject SCALEDOBJECT_NAME -n NAMESPACE -o jsonpath={.metadata.labels} +``` + +Once you have the selector, you have to add it to your query string as following: + +```bash +kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/sample-ns/s1-rabbitmq-queueName2?labelSelector=scaledobject.keda.sh%2Fname%3D{ScaledObjectName}" +``` \ No newline at end of file diff --git a/content/docs/2.7/operate/prometheus.md b/content/docs/2.7/operate/prometheus.md new file mode 100644 index 00000000000..b81ade21832 --- /dev/null +++ b/content/docs/2.7/operate/prometheus.md @@ -0,0 +1,16 @@ ++++ +title = "Integrate with Prometheus" +description = "Overview of all Prometheus metrics that KEDA provides" +weight = 200 ++++ + +## Prometheus Exporter Metrics + +The KEDA Metrics Adapter exposes Prometheus metrics which can be scraped on port `9022` (this can be changed by setting the `metrics-port` argument for the Metrics Adapter) at `/metrics`. The metrics collected in the Metrics Adapter are only active when the HPA is active (> 0 replicas). + +The following metrics are being gathered: + +- `keda_metrics_adapter_scaler_error_totals` - The total number of errors encountered for all scalers. +- `keda_metrics_adapter_scaled_object_error_totals`- The number of errors that have occurred for each scaled object. +- `keda_metrics_adapter_scaler_errors` - The number of errors that have occurred for each scaler. +- `keda_metrics_adapter_scaler_metrics_value`- The current value for each scaler's metric that would be used by the HPA in computing the target average. diff --git a/content/docs/2.7/scalers/_index.md b/content/docs/2.7/scalers/_index.md new file mode 100644 index 00000000000..6e74289235d --- /dev/null +++ b/content/docs/2.7/scalers/_index.md @@ -0,0 +1,6 @@ ++++ +title = "Scalers" +weight = 2 ++++ + +KEDA **scalers** can both detect if a deployment should be activated or deactivated, and feed custom metrics for a specific event source. diff --git a/content/docs/2.7/scalers/activemq.md b/content/docs/2.7/scalers/activemq.md new file mode 100644 index 00000000000..07b214ea91f --- /dev/null +++ b/content/docs/2.7/scalers/activemq.md @@ -0,0 +1,87 @@ ++++ +title = "ActiveMQ" +availability = "v2.6+" +maintainer = "Community" +description = "Scale applications based on ActiveMQ Queue." +layout = "scaler" +go_file = "activemq_scaler" ++++ + +### Trigger Specification + +This specification describes the `activemq` trigger that scales based on a ActiveMQ Queue. + +```yaml +triggers: +- type: activemq + metadata: + managementEndpoint: "activemq.activemq-test:8161" + destinationName: "testQueue" + brokerName: "activemq_broker" + targetQueueSize: "100" +``` + +**Parameter list:** + +- `managementEndpoint` - ActiveMQ management endpoint in format: `:`. +- `destinationName` - Name of the queue to check for the message count. +- `brokerName` - Name of the broker as defined in ActiveMQ. +- `targetQueueSize` - Target value for queue length passed to the scaler. The scaler will cause the replicas to increase if the queue message count is greater than the target value per active replica. (Default: `10`, Optional) +- `restAPITemplate` - Template to build REST API url to get queue size. (Default: `"http://{{.ManagementEndpoint}}/api/jolokia/read/org.apache.activemq:type=Broker,brokerName={{.BrokerName}},destinationType=Queue,destinationName={{.DestinationName}}/QueueSize"`, Optional) + +**Parameter Requirements:** + +- In case of `restAPITemplate` parameter is not used, parameters resolving the REST API Template are all **required**: `managementEndpoint`, `destinationName`, `brokerName`. +- ActiveMQ Scaler polls the ActiveMQ REST API to monitor message count of target queue. Currently, the scaler supports basic authentication. `username` and `password` are **required**. See [Authentication Parameters](#authentication-parameters) below. + +### Authentication Parameters + +You can authenticate by using username and password via `TriggerAuthentication` configuration. + +**Username and Password based Authentication:** + +- `username` - Username for connect to the management endpoint of ActiveMQ. +- `password` - Password for connect to the management endpoint of ActiveMQ. + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: activemq-secret +type: Opaque +data: + activemq-password: ACTIVEMQ_PASSWORD + activemq-username: ACTIVEMQ_USERNAME +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: trigger-auth-activemq +spec: + secretTargetRef: + - parameter: username + name: activemq-secret + key: activemq-username + - parameter: password + name: activemq-secret + key: activemq-password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: activemq-scaledobject +spec: + scaleTargetRef: + name: nginx-deployment + triggers: + - type: activemq + metadata: + managementEndpoint: "activemq.activemq-test:8161" + destinationName: "testQ" + brokerName: "localhost" + targetQueueSize: "50" + authenticationRef: + name: trigger-auth-activemq +``` diff --git a/content/docs/2.7/scalers/apache-kafka.md b/content/docs/2.7/scalers/apache-kafka.md new file mode 100644 index 00000000000..1b96874619e --- /dev/null +++ b/content/docs/2.7/scalers/apache-kafka.md @@ -0,0 +1,180 @@ ++++ +title = "Apache Kafka" +layout = "scaler" +availability = "v1.0+" +maintainer = "Microsoft" +description = "Scale applications based on an Apache Kafka topic or other services that support Kafka protocol." +go_file = "kafka_scaler" ++++ + +> **Notice:** +> - By default, the number of replicas will not exceed: +> - The number of partitions on a topic when a topic is specified; +> - The number of partitions of *all topics* in the consumer group when no topic is specified; +> +> That is, if `maxReplicaCount` is set more than number of partitions, the scaler won't scale up to target maxReplicaCount. See `allowIdleConsumers` below to disable this default behavior. +> - This is so because if there are more number of consumers than the number of partitions in a topic, then extra consumer will have to sit idle. + +### Trigger Specification + +This specification describes the `kafka` trigger for an Apache Kafka topic. + +```yaml +triggers: +- type: kafka + metadata: + bootstrapServers: kafka.svc:9092 + consumerGroup: my-group + topic: test-topic + lagThreshold: '5' + offsetResetPolicy: latest + allowIdleConsumers: false + version: 1.0.0 +``` + +**Parameter list:** + +- `bootstrapServers` - Comma separated list of Kafka brokers "hostname:port" to connect to for bootstrap. +- `consumerGroup` - Name of the consumer group used for checking the offset on the topic and processing the related lag. +- `topic` - Name of the topic on which processing the offset lag. (Optional, see note below) +- `lagThreshold` - Average target value to trigger scaling actions. (Default: `5`, Optional) +- `offsetResetPolicy` - The offset reset policy for the consumer. (Values: `latest`, `earliest`, Default: `latest`, Optional) +- `allowIdleConsumers` - When set to `true`, the number of replicas can exceed the number of +partitions on a topic, allowing for idle consumers. (Default: `false`, Optional) +- `version` - Version of your Kafka brokers. See [samara](https://github.com/Shopify/sarama) version (Default: `1.0.0`, Optional) + +> **Note:** +> +> When `topic` is unspecified, total offset lag will be calculated with all topics within the consumer group. +> - When there are **active** consumer instances, _all topics_ includes: +> - Topics the consumer is *currently* subscribing to; +> - Topics that the consumer group *had prior commit history* (up to retention period for `__consumer_offset`, default to 7 days, see [KIP-186](https://cwiki.apache.org/confluence/display/KAFKA/KIP-186%3A+Increase+offsets+retention+default+to+7+days)); +> - When there are **no active** consumer instances, _all topics_ only includes topics that the consumer group *had prior commit history*; +> --- +> An edge case exists where scaling could be **effectively disabled**: +> - Consumer never makes a commit (no record in `__consumer_offset`); +> - and `ScaledObject` had `minReplicaCount` as 0; +> +> In such case, KEDA could scale the consumer down to 0 when there is no lag and won't be able scale up due to the topic could not be auto discovered. +> +> Fix for such case: +> - Set `minReplicaCount` > 0; +> - or use multiple triggers where one supplies `topic` to ensure lag for that topic will always be detected; +### Authentication Parameters + + You can use `TriggerAuthentication` CRD to configure the authenticate by providing `sasl`, `username` and `password`, in case your Kafka cluster has SASL authentication turned on. If TLS is required you should set `tls` to `enable`. If required for your Kafka configuration, you may also provide a `ca`, `cert` and `key`. `cert` and `key` must be specified together. + +**Credential based authentication:** + +**SASL:** + +- `sasl` - Kafka SASL auth mode. (Values: `plaintext`, `scram_sha256` or `scram_sha512`, `none`, Default: `none`, Optional) +- `username` - Username used for sasl authentication. (Optional) +- `password` - Password used for sasl authentication. (Optional) + +**TLS:** + +- `tls` - To enable SSL auth for Kafka, set this to `enable`. If not set, TLS for Kafka is not used. (Values: `true`, `false`, Default: `false`, Optional) +- `ca` - Certificate authority file for TLS client authentication. (Optional) +- `cert` - Certificate for client authentication. (Optional) +- `key` - Key for client authentication. (Optional) + +### New Consumers and Offset Reset Policy + +When a new Kafka consumer is created, it must determine its consumer group initial position, i.e. the offset it will start to read from. The position is decided in Kafka consumers via a parameter `auto.offset.reset` and the possible values to set are `latest` (Kafka default), and `earliest`. This parameter in KEDA should be set accordingly. In this initial status, no offset has been committed to Kafka for the consumer group and any request for offset metadata will return an `INVALID_OFFSET`; so KEDA has to manage the consumer pod's autoscaling in relation to the offset reset policy that has been specified in the parameters: + +- If the policy is set to `earliest` (a new consumer wants to replay everything in the topic from its beginning) and no offset is committed, the scaler will return a lag value equal to the last offset in the topic (in the case of a new topic, 0), so it will scale the deployment to 0 replicas. If a new message is produced to the topic, KEDA will return the new value of the offset (1), and will scale the deployments to consume the message. +- If the policy is set to `latest` (so the new consumer will only consume new messages) and no offset is committed, the scaler will return a negative lag value, and will also tell the HPA to remain `active`, hence the deployment should have the minimum number of replicas running. This is to allow the consumer to read any new message on the topic, and commit its offset. + +### Example + +Your kafka cluster no SASL/TLS auth: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: kafka-scaledobject + namespace: default +spec: + scaleTargetRef: + name: azure-functions-deployment + pollingInterval: 30 + triggers: + - type: kafka + metadata: + bootstrapServers: localhost:9092 + consumerGroup: my-group # Make sure that this consumer group name is the same one as the one that is consuming topics + topic: test-topic + # Optional + lagThreshold: "50" + offsetResetPolicy: latest +``` + +Your kafka cluster turn on SASL/TLS auth: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-kafka-secrets + namespace: default +data: + sasl: "plaintext" + username: "admin" + password: "admin" + tls: "enable" + ca: + cert: + key: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-kafka-credential + namespace: default +spec: + secretTargetRef: + - parameter: sasl + name: keda-kafka-secrets + key: sasl + - parameter: username + name: keda-kafka-secrets + key: username + - parameter: password + name: keda-kafka-secrets + key: password + - parameter: tls + name: keda-kafka-secrets + key: tls + - parameter: ca + name: keda-kafka-secrets + key: ca + - parameter: cert + name: keda-kafka-secrets + key: cert + - parameter: key + name: keda-kafka-secrets + key: key +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: kafka-scaledobject + namespace: default +spec: + scaleTargetRef: + name: azure-functions-deployment + pollingInterval: 30 + triggers: + - type: kafka + metadata: + bootstrapServers: localhost:9092 + consumerGroup: my-group # Make sure that this consumer group name is the same one as the one that is consuming topics + topic: test-topic + # Optional + lagThreshold: "50" + offsetResetPolicy: latest + authenticationRef: + name: keda-trigger-auth-kafka-credential +``` diff --git a/content/docs/2.7/scalers/artemis.md b/content/docs/2.7/scalers/artemis.md new file mode 100644 index 00000000000..342dd6da099 --- /dev/null +++ b/content/docs/2.7/scalers/artemis.md @@ -0,0 +1,94 @@ ++++ +title = "ActiveMQ Artemis" +layout = "scaler" +availability = "v1.5+" +maintainer = "Community" +description = "Scale applications based on ActiveMQ Artemis queues" +go_file = "artemis_scaler" ++++ + +### Trigger Specification + +This specification describes the `artemis-queue` trigger for ActiveMQ Artemis queues. + +```yaml +triggers: +- type: artemis-queue + metadata: + managementEndpoint: "artemis-activemq.artemis:8161" + queueName: "test" + brokerName: "artemis-activemq" + brokerAddress: "test" + queueLength: '10' + username: 'ARTEMIS_USERNAME' + password: 'ARTEMIS_PASSWORD' + restApiTemplate: # Optional. Default : "http://<>/console/jolokia/read/org.apache.activemq.artemis:broker=\"<>\",component=addresses,address=\"<>\",subcomponent=queues,routing-type=\"anycast\",queue=\"<>\"/MessageCount" +``` + +**Parameter list:** + +- `managementEndpoint` - ActiveMQ Artemis management endpoint to connect to in `:` format. +- `queueName` - Name of the queue to check for the number of messages available. +- `brokerName` - Name of the broker as defined in Artemis. +- `brokerAddress` - Address of the broker. +- `queueLength` - Target value for queue length passed to the scaler. Example: if one pod can handle 10 messages, set the queue length target to 10. If the actual number of messages in the queue is 30, the scaler scales to 3 pods. (default: 10) +- `restApiTemplate` - Template to build REST API url to get queue size. (Default: `"http://<>/console/jolokia/read/org.apache.activemq.artemis:broker=\"<>\",component=addresses,address=\"<>\",subcomponent=queues,routing-type=\"anycast\",queue=\"<>\"/MessageCount"`, Optional) + +### Authentication Parameters + + You can use `TriggerAuthentication` CRD to configure the `username` and `password` to connect to the management endpoint. + +**Username and Password based authentication:** + +- `username` - The username to use to connect to the broker's management endpoint. +- `password` - The password to use to connect to the broker's management endpoint. + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: kedartemis + namespace: kedartemis + labels: + app: kedartemis +type: Opaque +data: + artemis-password: "YXJ0ZW1pcw==" + artemis-username: "YXJ0ZW1pcw==" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: trigger-auth-kedartemis + namespace: kedartemis +spec: + secretTargetRef: + - parameter: username + name: kedartemis + key: artemis-username + - parameter: password + name: kedartemis + key: artemis-password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: kedartemis-consumer-scaled-object + namespace: kedartemis +spec: + scaleTargetRef: + name: kedartemis-consumer + triggers: + - type: artemis-queue + metadata: + managementEndpoint: "artemis-activemq.artemis:8161" + queueName: "test" + queueLength: "50" + brokerName: "artemis-activemq" + brokerAddress: "test" + restApiTemplate: # Optional. Default: "http://<>/console/jolokia/read/org.apache.activemq.artemis:broker=\"<>\",component=addresses,address=\"<>\",subcomponent=queues,routing-type=\"anycast\",queue=\"<>\"/MessageCount" + authenticationRef: + name: trigger-auth-kedartemis +``` \ No newline at end of file diff --git a/content/docs/2.7/scalers/aws-cloudwatch.md b/content/docs/2.7/scalers/aws-cloudwatch.md new file mode 100644 index 00000000000..0101b9118f1 --- /dev/null +++ b/content/docs/2.7/scalers/aws-cloudwatch.md @@ -0,0 +1,130 @@ ++++ +title = "AWS CloudWatch" +layout = "scaler" +availability = "v1.0+" +maintainer = "Community" +description = "Scale applications based on AWS CloudWatch." +go_file = "aws_cloudwatch_scaler" ++++ + +### Trigger Specification + +This specification describes the `aws-cloudwatch` trigger that scales based on a AWS CloudWatch. + +```yaml +triggers: +- type: aws-cloudwatch + metadata: + # Required: namespace + namespace: AWS/SQS + # Required: Dimension Name - Supports specifying multiple dimension names by using ";" as a separator i.e. dimensionName: QueueName;QueueName + dimensionName: QueueName + # Required: Dimension Value - Supports specifying multiple dimension values by using ";" as a separator i.e. dimensionValue: queue1;queue2 + dimensionValue: keda + metricName: ApproximateNumberOfMessagesVisible + targetMetricValue: "2" + minMetricValue: "0" + # Required: region + awsRegion: "eu-west-1" + # Optional: AWS Access Key ID, can use TriggerAuthentication as well + awsAccessKeyIDFromEnv: AWS_ACCESS_KEY_ID # default AWS_ACCESS_KEY_ID + # Optional: AWS Secret Access Key, can use TriggerAuthentication as well + awsSecretAccessKeyFromEnv: AWS_SECRET_ACCESS_KEY # default AWS_SECRET_ACCESS_KEY + identityOwner: pod | operator # Optional. Default: pod + # Optional: Collection Time + metricCollectionTime: "300" # default 300 + # Optional: Metric Statistic + metricStat: "Average" # default "Average" + # Optional: Metric Statistic Period + metricStatPeriod: "300" # default 300 + # Optional: Metric Unit + metricUnit: "Count" # default "" + # Optional: Metric EndTime Offset + metricEndTimeOffset: "60" # default 0 +``` + +**Parameter list:** + +- `identityOwner` - Receive permissions on the CloudWatch via Pod Identity or from the KEDA operator itself (see below). (Values: `pod`, `operator`, Default: `pod`, Optional) + +> When `identityOwner` set to `operator` - the only requirement is that the KEDA operator has the correct IAM permissions on the CloudWatch. Additional Authentication Parameters are not required. + +- `metricCollectionTime` - How long in the past (seconds) should the scaler check AWS Cloudwatch. Used to define **StartTime** ([official documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html)). The value of `metricCollectionTime` must be greater than the `metricStatPeriod`, providing a value which is a multiply of the `metricStatPeriod` can improve performance on fetching data from Cloudwatch. In pratice setting `metricCollectionTime` 2-to-3 times more than the `metricStatPeriod` value can make sure the scaler is able to get data points back from Cloudwatch, the scaler will always use the most up-to-date datapoint if more datapoints are returned. (Default: `300`, Optional) +- `metricStat` - Which statistics metric is going to be used by the query. Used to define **Stat** ([official documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Statistic)). (Default: `Average`, Optional) +- `metricStatPeriod` - Which frequency is going to be used by the related query. Used to define **Period**. The value cannot be an arbitrary number, that it must be supported by Cloudwatch. More details can be found from ([official documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#CloudWatchPeriods)). (Default: `300`, Optional) +- `metricUnit` - Which unit is going to be used by the query. Used to define **Unit** ([official documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Unit)). (Default: `Average`, Optional) +- `metricEndTimeOffset` - How long in seconds to offset the **EndTime** ([official documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html)). Due to the eventual consistency model which is used by Cloudwatch, the latest datapoint one can get from Cloudwatch might not be accurate. The `metricEndTimeOffset` config provides a way to skip the most recent datapoint if needed. (Default: `0`, Optional) + + +### Authentication Parameters + +> These parameters are relevant only when `identityOwner` is set to `pod`. + +You can use `TriggerAuthentication` CRD to configure the authenticate by providing either a role ARN or a set of IAM credentials. + +**Pod identity based authentication:** + +- `podIdentity.provider` - Needs to be set to either `aws-kiam` or `aws-eks` on the `TriggerAuthentication` and the pod/service account must be configured correctly for your pod identity provider. + +**Role based authentication:** + +- `awsRoleArn` - Amazon Resource Names (ARNs) uniquely identify AWS resource. + +**Credential based authentication:** + +- `awsAccessKeyID` - Id of the user. +- `awsSecretAccessKey` - Access key for the user to authenticate with. +- `awsSessionToken` - Session token, only required when using [temporary credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html). + +The user will need access to read data from AWS CloudWatch. + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: test-secrets +data: + AWS_ACCESS_KEY_ID: # Required. + AWS_SECRET_ACCESS_KEY: # Required. + AWS_SESSION_TOKEN: # Required when using temporary credentials. +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-aws-credentials + namespace: keda-test +spec: + secretTargetRef: + - parameter: awsAccessKeyID # Required. + name: test-secrets # Required. + key: AWS_ACCESS_KEY_ID # Required. + - parameter: awsSecretAccessKey # Required. + name: test-secrets # Required. + key: AWS_SECRET_ACCESS_KEY # Required. + - parameter: awsSessionToken # Required when using temporary credentials. + name: test-secrets # Required when using temporary credentials. + key: AWS_SESSION_TOKEN # Required when using temporary credentials. +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: aws-cloudwatch-queue-scaledobject + namespace: keda-test +spec: + scaleTargetRef: + name: nginx-deployment + triggers: + - type: aws-cloudwatch + metadata: + namespace: AWS/SQS + dimensionName: QueueName + dimensionValue: keda + metricName: ApproximateNumberOfMessagesVisible + targetMetricValue: "2" + minMetricValue: "0" + awsRegion: "eu-west-1" + authenticationRef: + name: keda-trigger-auth-aws-credentials +``` diff --git a/content/docs/2.7/scalers/aws-kinesis.md b/content/docs/2.7/scalers/aws-kinesis.md new file mode 100644 index 00000000000..f27c14f03e0 --- /dev/null +++ b/content/docs/2.7/scalers/aws-kinesis.md @@ -0,0 +1,105 @@ ++++ +title = "AWS Kinesis Stream" +layout = "scaler" +availability = "v1.1+" +maintainer = "Community" +description = "Scale applications based on AWS Kinesis Stream." +go_file = "aws_kinesis_stream_scaler" ++++ + +### Trigger Specification + +This specification describes the `aws-kinesis-stream` trigger that scales based on the shard count of AWS Kinesis Stream. + +```yaml +triggers: +- type: aws-kinesis-stream + metadata: + # Required + streamName: myKinesisStream + # Required + awsRegion: "eu-west-1" + # Optional: Default: 2 + shardCount: "2" + identityOwner: pod | operator # Optional. Default: pod +``` + +**Parameter list:** + +- `streamName` - Name of AWS Kinesis Stream. +- `shardCount` - The target value that a Kinesis data streams consumer can handle. (Default: `2`, Optional) +- `awsRegion` - AWS Region for the Kinesis Stream. +- `identityOwner` - Receive permissions on the Kinesis Stream via Pod Identity or from the KEDA operator itself (see below). (Values: `pod`, `operator`, Default: `pod`, Optional) + +> When `identityOwner` set to `operator` - the only requirement is that the KEDA operator has the correct IAM permissions on the Kinesis Stream. Additional Authentication Parameters are not required. + +### Authentication Parameters + +> These parameters are relevant only when `identityOwner` is set to `pod`. + +You can use `TriggerAuthentication` CRD to configure the authenticate by providing either a role ARN or a set of IAM credentials, or use other [KEDA supported authentication methods](https://keda.sh/concepts/authentication). + +#### Delegate auth with TriggerAuthentication + +**Role based authentication:** + +- `awsRoleArn` - Amazon Resource Names (ARNs) uniquely identify AWS resource. + +**Credential based authentication:** + +- `awsAccessKeyID` - Id of the user. +- `awsSecretAccessKey` - Access key for the user to authenticate with. +- `awsSessionToken` - Session token, only required when using [temporary credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html). + +The user will need `DescribeStreamSummary` IAM permission policy to read data from AWS Kinesis Streams. + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: test-secrets + namespace: keda-test +data: + AWS_ACCESS_KEY_ID: # Required. + AWS_SECRET_ACCESS_KEY: # Required. + AWS_SESSION_TOKEN: # Required when using temporary credentials. +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-aws-credentials + namespace: keda-test +spec: + secretTargetRef: + - parameter: awsAccessKeyID # Required. + name: test-secrets # Required. + key: AWS_ACCESS_KEY_ID # Required. + - parameter: awsSecretAccessKey # Required. + name: test-secrets # Required. + key: AWS_SECRET_ACCESS_KEY # Required. + - parameter: awsSessionToken # Required when using temporary credentials. + name: test-secrets # Required when using temporary credentials. + key: AWS_SESSION_TOKEN # Required when using temporary credentials. +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: aws-kinesis-stream-scaledobject + namespace: keda-test +spec: + scaleTargetRef: + name: nginx-deployment + triggers: + - type: aws-kinesis-stream + authenticationRef: + name: keda-trigger-auth-aws-credentials + metadata: + # Required + streamName: myKinesisStream + # Required + awsRegion: "eu-west-1" + # Optional: Default: 2 + shardCount: "2" +``` diff --git a/content/docs/2.7/scalers/aws-sqs.md b/content/docs/2.7/scalers/aws-sqs.md new file mode 100644 index 00000000000..76163bd8aa5 --- /dev/null +++ b/content/docs/2.7/scalers/aws-sqs.md @@ -0,0 +1,179 @@ ++++ +title = "AWS SQS Queue" +layout = "scaler" +availability = "v1.0+" +maintainer = "Community" +description = "Scale applications based on AWS SQS Queue." +go_file = "aws_sqs_queue_scaler" ++++ + +### Trigger Specification + +This specification describes the `aws-sqs-queue` trigger that scales based on an AWS SQS Queue. + +```yaml +triggers: +- type: aws-sqs-queue + metadata: + # Required: queueURL + queueURL: https://sqs.eu-west-1.amazonaws.com/account_id/QueueName + queueLength: "5" # Default: "5" + # Required: awsRegion + awsRegion: "eu-west-1" + identityOwner: pod | operator # Optional. Default: pod +``` + +**Parameter list:** + +- `queueURL` - Full URL for the SQS Queue. The simple name of the queue can be used in case there's no ambiguity. +- `queueLength` - Target value for queue length passed to the scaler. Example: if one pod can handle 10 messages, set the queue length target to 10. If the actual messages in the SQS Queue is 30, the scaler scales to 3 pods. (default: 5) + +> For the purposes of scaling, "actual messages" is equal to `ApproximateNumberOfMessages` + `ApproximateNumberOfMessagesNotVislble`, since `NotVisible` in SQS terms means the message is still in-flight/processing. + +- `awsRegion` - AWS Region for the SQS Queue. +- `identityOwner` - Receive permissions on the SQS Queue via Pod Identity or from the KEDA operator itself (see below). (Values: `pod`, `operator`, Default: `pod`, Optional) + +> When `identityOwner` set to `operator` - the only requirement is that the KEDA operator has the correct IAM permissions on the SQS queue. Additional Authentication Parameters are not required. + +### Authentication Parameters + +> These parameters are relevant only when `identityOwner` is set to `pod`. + +You can use `TriggerAuthentication` CRD to configure the authenticate by providing either a role ARN or a set of IAM credentials. + +**Pod identity based authentication:** + +- `podIdentity.provider` - Needs to be set to either `aws-kiam` or `aws-eks` on the `TriggerAuthentication` and the pod/service account must be configured correctly for your pod identity provider. + +**Role based authentication:** + +- `awsRoleArn` - Amazon Resource Names (ARNs) uniquely identify AWS resource. + +**Credential based authentication:** + +- `awsAccessKeyID` - Id of the user. +- `awsSecretAccessKey` - Access key for the user to authenticate with. +- `awsSessionToken` - Session token, only required when using [temporary credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html). + +The user will need access to read properties from the specified AWS SQS queue. + +### Example + +#### Scaling a deployment using podIdentity providers + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-aws-credentials + namespace: keda-test +spec: + podIdentity: + provider: aws-kiam # or aws-eks when using IRSA +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: aws-sqs-queue-scaledobject + namespace: keda-test +spec: + scaleTargetRef: + name: nginx-deployment + triggers: + - type: aws-sqs-queue + authenticationRef: + name: keda-trigger-auth-aws-credentials + metadata: + queueURL: myQueue + queueLength: "5" + awsRegion: "eu-west-1" +``` + +#### Scaling a deployment using IAM Role + +When you need to specify the IAM Role used to access the sqs queue. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: test-secrets +data: + AWS_ROLE_ARN: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-aws-credentials + namespace: keda-test +spec: + secretTargetRef: + - parameter: awsRoleArn # The property in KEDA. + name: test-secrets # The name of the kubernetes secret. + key: AWS_ROLE_ARN # The key from the kubernetes secret. +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: aws-sqs-queue-scaledobject + namespace: keda-test +spec: + scaleTargetRef: + name: nginx-deployment + triggers: + - type: aws-sqs-queue + authenticationRef: + name: keda-trigger-auth-aws-credentials + metadata: + queueURL: myQueue + queueLength: "5" + awsRegion: "eu-west-1" +``` + + +#### Scaling a deployment using IAM Users + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: test-secrets +data: + AWS_ACCESS_KEY_ID: # Required. + AWS_SECRET_ACCESS_KEY: # Required. + AWS_SESSION_TOKEN: # Required when using temporary credentials. +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-aws-credentials + namespace: keda-test +spec: + secretTargetRef: + - parameter: awsAccessKeyID # Required. + name: test-secrets # Required. + key: AWS_ACCESS_KEY_ID # Required. + - parameter: awsSecretAccessKey # Required. + name: test-secrets # Required. + key: AWS_SECRET_ACCESS_KEY # Required. + - parameter: awsSessionToken # Required when using temporary credentials. + name: test-secrets # Required when using temporary credentials. + key: AWS_SESSION_TOKEN # Required when using temporary credentials. +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: aws-sqs-queue-scaledobject + namespace: keda-test +spec: + scaleTargetRef: + name: nginx-deployment + triggers: + - type: aws-sqs-queue + authenticationRef: + name: keda-trigger-auth-aws-credentials + metadata: + queueURL: myQueue + queueLength: "5" + awsRegion: "eu-west-1" +``` diff --git a/content/docs/2.7/scalers/azure-app-insights.md b/content/docs/2.7/scalers/azure-app-insights.md new file mode 100644 index 00000000000..7e91ad47610 --- /dev/null +++ b/content/docs/2.7/scalers/azure-app-insights.md @@ -0,0 +1,201 @@ ++++ +title = "Azure Application Insights" +layout = "scaler" +availability = "v2.6+" +maintainer = "Community" +description = "Scale applications based on Azure Application Insights metrics." +go_file = "azure_app_insights_scaler" ++++ + +### Trigger Specification + +This specification describes the `azure-app-insights` trigger that scales based on an Azure Application Insights metric. + +```yaml +triggers: +- type: azure-app-insights + metadata: + metricAggregationTimespan: "0:1" + metricAggregationType: avg + metricFilter: cloud/roleName eq 'role_name' + metricId: "customMetrics/example-metric" + targetValue: "1" + activeDirectoryClientIdFromEnv: CLIENT_ID_ENV_NAME # Optional, can use TriggerAuthentication as well + activeDirectoryClientPasswordFromEnv: CLIENT_PASSWORD_ENV_NAME # Optional, can use TriggerAuthentication as well + applicationInsightsIdFromEnv: APP_ID # Optional, can use TriggerAuthentication as well + tenantIdFromEnv: TENANT_ID` # Optional, can use TriggerAuthentication as well +``` + +This scaler is backed by the Azure Application Instance REST API. Please see [this](https://docs.microsoft.com/en-us/rest/api/application-insights/metrics/get) page +for further details. + +**Parameter list:** + +- `tenantId` - Id of the tenant that contains the Azure resource. This is used for authentication. +- `metricId` - The name of the Application Insights metric to query. Use the [Azure Command Line Interface](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) to run `az monitor app-insights metrics get-metadata` to see a list of available metrics. +- `targetValue` - Target value to trigger scaling actions. +- `metricAggregationType` - Aggregation method of the Azure Application Insights metric. The aggregation methods vary from metric to metric. The `az monitor app-insights metrics get-metadata` command can be used to determine which methods apply to a given metric. (Some common aggregation methods are `avg`, `count`, `sum`, `min`, and `max`) +- `metricAggregationInterval` - Collection time of the metric in format `"hh:mm"`. +- `applicationInsightsId` - Id of the Application Insights instance to query. This is a GUID that can be retrieved from the Application Insight's `API Access` blade in the Azure Portal. +- `activeDirectoryClientId` - Id of the Active Directory client. The client must have `Monitoring Reader` permissions for the Application Insights instance. +- `activeDirectoryClientPassword` - Password of the Active Directory client password. +- `metricFilter` - Further specify the metrics query using a filter. For example `cloud/roleName eq 'example`. (Optional) + +Some parameters can be provided using environment variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `activeDirectoryClientIdFromEnv` - Name of the environment variable that contains the Id of the Active Directory application. (Optional) +- `activeDirectoryClientPasswordFromEnv` - Name of the environment variable that contains the Active Directory client password. (Optional) +- `applicationInsightsIdFromEnv` - Name of the environment variable that contains the Application Insights Id. (Optional) +- `tenantIdFromEnv` - Name of the environment variable that contains the Id of the tenant that contains the Application Insights instance. (Optional) + +### Authentication Parameters + +You can use the `TriggerAuthentication` CRD to configure the authentication by providing a set of Azure Active Directory credentials or by using pod identity. + +**Credential based authentication:** + +- `activeDirectoryClientId` - Id of the Active Directory application which requires at least `Monitoring Reader` permissions. +- `activeDirectoryClientPassword` - Password of the Active Directory application. +- `applicationInsightsId` - Id of the Application Insights instance to query. +- `tenantId` - Id of the tenant that contains the Azure resource. + +The principal will need `Monitoring Reader` access to query metrics from the Application Insights instance. + +**Pod identity based authentication:** + +[Azure Active Directory pod-managed identity](https://docs.microsoft.com/en-us/azure/aks/use-azure-ad-pod-identity) can be used +in place of credential based authentication. The following section contains an example of a `TriggerAuthentication` using pod identity. + +### Example + +The following example illustrates the use of a TriggerAuthentication to connect to Application Insights. +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: azure-app-insights-secrets +data: + activeDirectoryClientId: + activeDirectoryClientPassword: + applicationInsightsId: + tenantId: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: azure-app-insights-trigger-auth +spec: + secretTargetRef: + - parameter: activeDirectoryClientId + name: azure-app-insights-secrets + key: activeDirectoryClientId + - parameter: activeDirectoryClientPassword + name: azure-app-insights-secrets + key: activeDirectoryClientPassword + - parameter: applicationInsightsId + name: azure-app-insights-secrets + key: applicationInsightsId + - parameter: tenantId + name: azure-app-insights-secrets + key: tenantId + # or Pod Identity, kind: Secret is not required in case of pod Identity + podIdentity: + provider: azure +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: azure-app-insights-scaler +spec: + scaleTargetRef: + name: azure-app-insights-example + minReplicaCount: 1 + maxReplicaCount: 10 + triggers: + - type: azure-app-insights + metadata: + metricId: "customMetrics/example-metric" + metricAggregationTimespan: "0:5" + metricAggregationType: avg + metricFilter: cloud/roleName eq 'example' + targetValue: "1" + authenticationRef: + name: azure-app-insights-trigger-auth +``` + +The following example illustrates the use of environment variables to connect to Application Insights. +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: azure-app-insights-secrets +type: Opaque +data: + activeDirectoryClientId: + activeDirectoryClientPassword: + applicationInsightsId: + tenantId: +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: azure-app-insights-example +spec: + replicas: 0 + selector: + matchLabels: + app: azure-app-insights-example + template: + metadata: + labels: + app: azure-app-insights-example + spec: + containers: + - name: example + image: nginx:1.16.1 + env: + - name: ACTIVE_DIRECTORY_ID + valueFrom: + secretKeyRef: + name: azure-app-insights-secrets + key: activeDirectoryClientId + - name: ACTIVE_DIRECTORY_PASSWORD + valueFrom: + secretKeyRef: + name: azure-app-insights-secrets + key: activeDirectoryClientPassword + - name: APP_INSIGHTS_APP_ID + valueFrom: + secretKeyRef: + name: azure-app-insights-secrets + key: applicationInsightsId + - name: TENANT_ID + valueFrom: + secretKeyRef: + name: azure-app-insights-secrets + key: tenantId +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: azure-app-insights-scaler +spec: + scaleTargetRef: + name: azure-app-insights-example + pollingInterval: 5 + cooldownPeriod: 5 + minReplicaCount: 0 + maxReplicaCount: 2 + triggers: + - type: azure-app-insights + metadata: + metricId: "customMetrics/example-metric" + metricAggregationTimespan: "0:5" + metricAggregationType: avg + metricFilter: cloud/roleName eq 'example' + targetValue: "1" + activeDirectoryClientIdFromEnv: ACTIVE_DIRECTORY_ID + activeDirectoryClientPasswordFromEnv: ACTIVE_DIRECTORY_PASSWORD + applicationInsightsIdFromEnv: APP_INSIGHTS_APP_ID + tenantIdFromEnv: TENANT_ID +``` diff --git a/content/docs/2.7/scalers/azure-event-hub.md b/content/docs/2.7/scalers/azure-event-hub.md new file mode 100644 index 00000000000..c2efe889a51 --- /dev/null +++ b/content/docs/2.7/scalers/azure-event-hub.md @@ -0,0 +1,88 @@ ++++ +title = "Azure Event Hubs" +layout = "scaler" +availability = "v1.0+" +maintainer = "Community" +description = "Scale applications based on Azure Event Hubs." +go_file = "azure_eventhub_scaler" ++++ + +### Trigger Specification + +This specification describes the `azure-eventhub` trigger for Azure Event Hubs. + +```yaml +triggers: +- type: azure-eventhub + metadata: + connectionFromEnv: EVENTHUB_CONNECTIONSTRING_ENV_NAME + storageConnectionFromEnv: STORAGE_CONNECTIONSTRING_ENV_NAME + consumerGroup: $Default + unprocessedEventThreshold: '64' + blobContainer: 'name_of_container' +``` + +**Parameter list:** + +- `connectionFromEnv` - Name of the environment variable your deployment uses to get the connection string appended with `EntityPath=`. +- `storageConnectionFromEnv` - Name of the environment variable that provides connection string for Azure Storage Account to store checkpoint. As of now the Event Hub scaler only reads from Azure Blob Storage. +- `consumerGroup` - Consumer group of Azure Event Hub consumer. (default: `$default`) +- `unprocessedEventThreshold` - Average target value to trigger scaling actions. (Default: `64`, Optional) +- `blobContainer` - Container name to store checkpoint. This is needed for every `checkpointStrategy` except of `AzureFunction`. With Azure Functions the `blobContainer` is autogenerated and cannot be overridden. +- `checkpointStrategy` - configure the checkpoint behaviour of different Event Hub SDKs. (default: `""`) + - `azureFunction` - Suitable for Azure Functions & Azure WebJobs SDK. This is the default setting, when `blobcontainer` is not specified. + - `blobMetadata` - For all implementations that store checkpoint information on blob metadata such as current C#, Python, Java and JavaScript Event Hub SDKs. + - `goSdk` - For all implementations using the [Golang SDK](https://github.com/Azure/azure-event-hubs-go)'s checkpointing, for example Dapr. + - When no checkpoint strategy is specified, the Event Hub scaler will use backwards compatibility and able to scale older implementations of C#, Python or Java Event Hub SDKs. (see "Legacy checkpointing"). If this behaviour should be used, `blobContainer` is also required. + +> 💡 **Legacy Checkpointing:** C# applications, which use the `Microsoft.Azure.EventHubs` package, Java applications which use the `azure-eventhubs-eph` package or Python applications which use the `azure-eventhub` below `v5` are supported if no `checkpointStrategy` is specified. These legacy implementations are based on the `EventProcessorHost` client, which stores checkpoint information as blob container content +> +> C# Applications that use the current `Azure.Messaging.EventHubs` package, Java applications which use the current `azure-messaging-eventhubs` package or Python applications since `azure-eventhub v5`, have to set the `checkpointStrategy`to `blobMetadata`. All these implementations based on the `EventProcessorClient` which stores checkpoint information as blob metadata. + +> 💡 The Azure Storage connection string is not compatible with connection string created from a Shared Access Signature. +### Authentication Parameters + +The common way of authenticating to Azure Event Hub is by using the connection string. However, you can use [Pod Identity](https://azure.github.io/aad-pod-identity/docs/demo/standard_walkthrough/) if you host your cluster in Azure AKS, and if have configured it to support Pod Identity. + +To use Pod Identity, you have to add a [TriggerAuthentication](https://keda.sh/docs/2.0/concepts/authentication/#re-use-credentials-and-delegate-auth-with-triggerauthentication) and configure it to use Pod Identity like so: + +``` +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: nameOfTriggerAuth + namespace: default +spec: + podIdentity: + provider: Azure +``` + +When you do so, the Event Hub scaler will depend on the existence of two configurations you have to provide: `eventHubNamespace` and `eventHubName`. + + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: azure-eventhub-scaledobject + namespace: default +spec: + scaleTargetRef: + name: azureeventhub-function + triggers: + - type: azure-eventhub + metadata: + # Required + storageConnectionFromEnv: AzureWebJobsStorage + # Required if not using Pod Identity + connectionFromEnv: EventHub + # Required if using Pod Identity + eventHubNamespace: AzureEventHubNameSpace + eventHubName: NameOfTheEventHub +# Optional + consumerGroup: $Default # default: $Default + unprocessedEventThreshold: '64' # default 64 events. + blobContainer: ehcontainer +``` diff --git a/content/docs/2.7/scalers/azure-log-analytics.md b/content/docs/2.7/scalers/azure-log-analytics.md new file mode 100644 index 00000000000..b8da96a0b90 --- /dev/null +++ b/content/docs/2.7/scalers/azure-log-analytics.md @@ -0,0 +1,369 @@ ++++ +title = "Azure Log Analytics" +layout = "scaler" +availability = "v2.0+" +maintainer = "Community" +description = "Scale applications based on Azure Log Analytics query result" +go_file = "azure_log_analytics_scaler" ++++ + +### Trigger Specification + +This specification describes the `azure-log-analytics` trigger for Azure Log Analytics query result. Here is an example of providing values in metadata: + +```yaml +triggers: + - type: azure-log-analytics + metadata: + tenantId: "AZURE_AD_TENANT_ID" + clientId: "SERVICE_PRINCIPAL_CLIENT_ID" + clientSecret: "SERVICE_PRINCIPAL_PASSWORD" + workspaceId: "LOG_ANALYTICS_WORKSPACE_ID" + query: | + let AppName = "web"; + let ClusterName = "demo-cluster"; + let AvgDuration = ago(10m); + let ThresholdCoefficient = 0.8; + Perf + | where InstanceName contains AppName + | where InstanceName contains ClusterName + | where CounterName == "cpuUsageNanoCores" + | where TimeGenerated > AvgDuration + | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1) + | summarize MetricValue=round(avg(CounterValue)) by CounterName, AppName + | join (Perf + | where InstanceName contains AppName + | where InstanceName contains ClusterName + | where CounterName == "cpuLimitNanoCores" + | where TimeGenerated > AvgDuration + | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1) + | summarize arg_max(TimeGenerated, *) by AppName, CounterName + | project Limit = CounterValue, TimeGenerated, CounterPath, AppName) + on AppName + | project MetricValue, Threshold = Limit * ThresholdCoefficient + threshold: "1900000000" + # Alternatively, you can use existing environment variables to read configuration from: + # See details in "Parameter list" section + workspaceIdFromEnv: LOG_ANALYTICS_WORKSPACE_ID_ENV_NAME # Optional. You can use this instead of `workspaceId` parameter. + clientIdFromEnv: SERVICE_PRINCIPAL_CLIENT_ID_ENV_NAME # Optional. You can use this instead of `clientId` parameter. + tenantIdFromEnv: AZURE_AD_TENANT_ID_ENV_NAME # Optional. You can use this instead of `tenantId` parameter. + clientSecretFromEnv: SERVICE_PRINCIPAL_PASSWORD_ENV_NAME # Optional. You can use this instead of `clientSecret` parameter. +``` + +**Parameter list:** + +- `tenantId` - Id of the Azure Active Directory tenant. Follow [this](https://docs.microsoft.com/en-us/cli/azure/account?view=azure-cli-latest#az-account-show) link to retrieve your tenant id. +- `clientId` - Id of the application from your Azure AD Application/service principal. Follow [this](https://docs.microsoft.com/en-us/cli/azure/ad/sp?view=azure-cli-latest) link to create your service principal. +- `clientSecret` - Password from your Azure AD Application/service principal. +- `workspaceId` - Id of Log Analytics workspace. Follow [this](https://docs.microsoft.com/en-us/cli/azure/monitor/log-analytics/workspace?view=azure-cli-latest#az-monitor-log-analytics-workspace-list) link to get your Log Analytics workspace id. +- `query` - Log Analytics [kusto](https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/get-started-queries) query, JSON escaped. You can use [this](https://www.freeformatter.com/json-escape.html) tool to convert your query from Log Analytics query editor to JSON escaped string, and then review YAML specific escapes. +- `threshold` - Value that is used as a threshold to calculate # of pods for scale target. +- `metricName` - Name to assign to the metric. (Optional, if not set KEDA will generate a name based on the workspaceId) + +The authentication parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `tenantIdFromEnv` - An environmental variable name, that stores Azure Active Directory tenant id. Follow [this](https://docs.microsoft.com/en-us/cli/azure/account?view=azure-cli-latest#az-account-show) link to retrieve your tenant id. (Optional) +- `clientIdFromEnv` - An environmental variable name, that stores Application id from your Azure AD Application/service principal. Follow [this](https://docs.microsoft.com/en-us/cli/azure/ad/sp?view=azure-cli-latest) link to create your service principal. (Optional) +- `clientSecretFromEnv` - An environmental variable name, that stores password from your Azure AD Application/service principal. (Optional) +- `workspaceIdFromEnv` - An environmental variable name, that stores your Log Analytics workspace id. Follow [this](https://docs.microsoft.com/en-us/cli/azure/monitor/log-analytics/workspace?view=azure-cli-latest#az-monitor-log-analytics-workspace-list) link to get your Log Analytics workspace id. (Optional) + +> 💡 **NOTE:** The workspaceID for Log Analytics is called the `customerId`; + it's not the full `id`! the example `az` command below can be used. + +```sh +az monitor log-analytics workspace list --query '[]. {ResourceGroup:resourceGroup,WorkspaceName:name,"workspaceID (customerId)":customerId}' -o table +``` + +### Query Guidance + +It is important to design your query to return 1 table with 1 row. A good practice is to add "| limit 1" at the end of your query. + +Scaler will take value from: + +- 1st cell as Metrics Value. +- 2d cell as Threshold (optional). + +You can define threshold in trigger metadata, it will be used if your query results only 1 cell, that will be interpreted as metric value. Be aware, even if you have defined threshold in metadata, it can be overwritten by your query. + +Data types of your query result should be: real, int or long. Other data types are not supported. Later, during runtime, your data will be converted to int64. + +Be careful with setting up "pollingInterval" and long running queries. Test your query before. + +Example query to get `MetricValue` and `Threshold` based on CPU usage and limits, defined for the pod. + +```kusto +let AppName = "web"; +let ClusterName = "demo-cluster"; +let AvgDuration = ago(10m); +let ThresholdCoefficient = 0.8; +Perf +| where InstanceName contains AppName +| where InstanceName contains ClusterName +| where CounterName == "cpuUsageNanoCores" +| where TimeGenerated > AvgDuration +| extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1) +| summarize MetricValue=round(avg(CounterValue)) by CounterName, AppName +| join (Perf + | where InstanceName contains AppName + | where InstanceName contains ClusterName + | where CounterName == "cpuLimitNanoCores" + | where TimeGenerated > AvgDuration + | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1) + | summarize arg_max(TimeGenerated, *) by AppName, CounterName + | project Limit = CounterValue, TimeGenerated, CounterPath, AppName) + on AppName +| project MetricValue, Threshold = Limit * ThresholdCoefficient +``` + +Example result: + +![Azure Log Analytics query example](/img/azure-log-analytics-scaler-query-example.png) + +### Scaler Limitations + +- As it was mentioned before, you can define a threshold using query (2d cell of query result will be interpret as threshold). Be aware! Threshold from query result will be set only once, during scaler creation. So, if your query will return different threshold values during runtime, they will not be propagated to Horizontal Pod Autoscaler target. + +### Authentication Parameters + + You can use `TriggerAuthentication` CRD to configure the authentication by providing a set of Azure Active Directory credentials and resource identifiers. + +**Service Principal based authentication:** + +- `tenantId` - Azure Active Directory tenant id. Follow [this](https://docs.microsoft.com/en-us/cli/azure/account?view=azure-cli-latest#az-account-show) link to retrieve your tenant id. +- `clientId` - Application id from your Azure AD Application/service principal. Follow [this](https://docs.microsoft.com/en-us/cli/azure/ad/sp?view=azure-cli-latest) link to create your service principal. +- `clientSecret` - Password from your Azure AD Application/service principal. +- `workspaceId` - Your Log Analytics workspace id. Follow [this](https://docs.microsoft.com/en-us/cli/azure/monitor/log-analytics/workspace?view=azure-cli-latest#az-monitor-log-analytics-workspace-list) link to get your Log Analytics workspace id. + +**Managed identity based authentication:** + +You can use managed identity to request access token for Log Analytics API. The advantage of this approach is that there is no need to store secrets in Kubernetes. Read [more](https://docs.microsoft.com/en-us/azure/aks/use-managed-identity) about managed identities in Azure Kubernetes Service. + +### Example + +#### Service Principal based authentication + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: kedaloganalytics + namespace: kedaloganalytics + labels: + app: kedaloganalytics +type: Opaque +data: + tenantId: "QVpVUkVfQURfVEVOQU5UX0lE" #Base64 encoded Azure Active Directory tenant id + clientId: "U0VSVklDRV9QUklOQ0lQQUxfQ0xJRU5UX0lE" #Base64 encoded Application id from your Azure AD Application/service principal + clientSecret: "U0VSVklDRV9QUklOQ0lQQUxfUEFTU1dPUkQ=" #Base64 encoded Password from your Azure AD Application/service principal + workspaceId: "TE9HX0FOQUxZVElDU19XT1JLU1BBQ0VfSUQ=" #Base64 encoded Log Analytics workspace id +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: trigger-auth-kedaloganalytics + namespace: kedaloganalytics +spec: + secretTargetRef: + - parameter: tenantId + name: kedaloganalytics + key: tenantId + - parameter: clientId + name: kedaloganalytics + key: clientId + - parameter: clientSecret + name: kedaloganalytics + key: clientSecret + - parameter: workspaceId + name: kedaloganalytics + key: workspaceId +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: kedaloganalytics-consumer-scaled-object + namespace: kedaloganalytics + labels: + deploymentName: kedaloganalytics-consumer +spec: + scaleTargetRef: + kind: #Optional: Default: Deployment, Available Options: ReplicaSet, Deployment, DaemonSet, StatefulSet + name: kedaloganalytics-consumer + pollingInterval: 30 + cooldownPeriod: 30 + minReplicaCount: 1 + maxReplicaCount: 10 + triggers: + - type: azure-log-analytics + metadata: + query: | + let AppName = "web"; + let ClusterName = "demo-cluster"; + let AvgDuration = ago(10m); + let ThresholdCoefficient = 0.8; + Perf + | where InstanceName contains AppName + | where InstanceName contains ClusterName + | where CounterName == "cpuUsageNanoCores" + | where TimeGenerated > AvgDuration + | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1) + | summarize MetricValue=round(avg(CounterValue)) by CounterName, AppName + | join (Perf + | where InstanceName contains AppName + | where InstanceName contains ClusterName + | where CounterName == "cpuLimitNanoCores" + | where TimeGenerated > AvgDuration + | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1) + | summarize arg_max(TimeGenerated, *) by AppName, CounterName + | project Limit = CounterValue, TimeGenerated, CounterPath, AppName) + on AppName + | project MetricValue, Threshold = Limit * ThresholdCoefficient + threshold: "1900000000" + authenticationRef: + name: trigger-auth-kedaloganalytics +``` + +#### Managed identity based authentication + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: trigger-auth-kedaloganalytics + namespace: kedaloganalytics +spec: + podIdentity: + provider: azure +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: kedaloganalytics-consumer-scaled-object + namespace: kedaloganalytics + labels: + deploymentName: kedaloganalytics-consumer +spec: + scaleTargetRef: + kind: #Optional: Default: Deployment, Available Options: ReplicaSet, Deployment, DaemonSet, StatefulSet + name: kedaloganalytics-consumer + pollingInterval: 30 + cooldownPeriod: 30 + minReplicaCount: 1 + maxReplicaCount: 10 + triggers: + - type: azure-log-analytics + metadata: + workspaceId: "81963c40-af2e-47cd-8e72-3002e08aa2af" + query: | + let AppName = "web"; + let ClusterName = "demo-cluster"; + let AvgDuration = ago(10m); + let ThresholdCoefficient = 0.8; + Perf + | where InstanceName contains AppName + | where InstanceName contains ClusterName + | where CounterName == "cpuUsageNanoCores" + | where TimeGenerated > AvgDuration + | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1) + | summarize MetricValue=round(avg(CounterValue)) by CounterName, AppName + | join (Perf + | where InstanceName contains AppName + | where InstanceName contains ClusterName + | where CounterName == "cpuLimitNanoCores" + | where TimeGenerated > AvgDuration + | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1) + | summarize arg_max(TimeGenerated, *) by AppName, CounterName + | project Limit = CounterValue, TimeGenerated, CounterPath, AppName) + on AppName + | project MetricValue, Threshold = Limit * ThresholdCoefficient + threshold: "1900000000" + authenticationRef: + name: trigger-auth-kedaloganalytics +``` + +### Guides + +#### Enabling managed identity authentication for Log Analytics scaler + +Use the following commands to create user defined identity, role assignment to Azure Log Analytics and deploy\update KEDA: + +```sh +export SUBSCRIPTION_ID="" +export RESOURCE_GROUP="" +export CLUSTER_NAME="" +export CLUSTER_LOCATION="" # "westeurope", "northeurope"... +export IDENTITY_NAME="" #Any name +export LOG_ANALYTICS_RESOURCE_ID="" + +# Login to Azure, set subscription, get AKS credentials +az login +az account set -s "${SUBSCRIPTION_ID}" +az aks get-credentials -n ${CLUSTER_NAME} -g ${RESOURCE_GROUP} + +# ------- Cluster preparation. Run this block only once for fresh cluster. +# Clone repo and run initial role assignment +git clone https://github.com/Azure/aad-pod-identity.git +./aad-pod-identity/hack/role-assignment.sh + +#Deploy aad-pod-identity using Helm 3 +helm repo add aad-pod-identity https://raw.githubusercontent.com/Azure/aad-pod-identity/master/charts +helm repo update +helm install aad-pod-identity aad-pod-identity/aad-pod-identity --namespace=kube-system +# ------------------------------------------------------------------------------------------- + +#Create identity +az identity create -g ${RESOURCE_GROUP} -n ${IDENTITY_NAME} +export IDENTITY_CLIENT_ID="$(az identity show -g ${RESOURCE_GROUP} -n ${IDENTITY_NAME} --query clientId -otsv)" +export IDENTITY_RESOURCE_ID="$(az identity show -g ${RESOURCE_GROUP} -n ${IDENTITY_NAME} --query id -otsv)" + +#Assign reader permissions for your identity to Log Analytics workspace +#WARNING: It can take some time while identity will be provisioned. +#If you see an error: "Principal SOME_ID does not exist in the directory SOME_ID", just wait couple of minutes and then retry. +az role assignment create --role "Log Analytics Reader" --assignee ${IDENTITY_CLIENT_ID} --scope ${LOG_ANALYTICS_RESOURCE_ID} + +# Allow cluster to control identity created earlier. +ID="$(az aks show -g ${RESOURCE_GROUP} -n ${CLUSTER_NAME} --query servicePrincipalProfile.clientId -otsv)" +if [[ "${ID:-}" == "msi" ]]; then + ID="$(az aks show -g ${RESOURCE_GROUP} -n ${CLUSTER_NAME} --query identityProfile.kubeletidentity.clientId -otsv)" +fi +az role assignment create --role "Managed Identity Operator" --assignee "${ID}" --scope "${IDENTITY_RESOURCE_ID}" + +# Create AzureIdentity and AzureIdentityBinding +cat < # Optional, can use TriggerAuthentication as well + activeDirectoryClientIdFromEnv: CLIENT_ID_ENV_NAME # Optional, can use TriggerAuthentication as well + activeDirectoryClientPasswordFromEnv: CLIENT_PASSWORD_ENV_NAME # Optional, can use TriggerAuthentication as well +``` + +**Parameter list:** + +- `resourceURI` - Shortened URI to the Azure resource with format `"//"`. +- `tenantId` - Id of the tenant that contains the Azure resource. This is used for authentication. +- `subscriptionId` - Id of Azure subscription that contains the Azure resource. This is used for determining the full resource URI. +- `resourceGroupName` - Name of the resource group for the Azure resource. +- `metricName` - Name of the metric to query. + - Azure metrics are available as a list in the [official documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/metrics-supported). + - Custom metric name when querying Azure Monitor [custom metrics](https://docs.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-custom-overview). +- `metricNamespace` - Name of the metric namespace. Required when `metricName` is a custom metric. +- `targetValue` - Target value to trigger scaling actions. +- `metricAggregationType` - Aggregation method of the Azure Monitor metric. Options include `Average`, `Total`, `Maximum` with a full list in the [official documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/metrics-supported). +- `metricFilter` - Name of the filter to be more specific by using dimensions listed in the [official documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/metrics-supported). (Optional) +- `metricAggregationInterval` - Collection time of the metric in format `"hh:mm:ss"` (Default: `"0:5:0"`, Optional) +- `activeDirectoryClientId` - Id of the Active Directory application which requires at least `Monitoring Reader` permissions. (Optional) +- `activeDirectoryClientPasswordFromEnv` - Name of the environment variable that contains the active directory client password. (Optional) + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `activeDirectoryClientIdFromEnv` - Id of the Active Directory application which requires at least `Monitoring Reader` permissions, similar to `activeDirectoryClientId`, but reads it from an environment variable on the scale target. (Optional) + +### Authentication Parameters + +You can use `TriggerAuthentication` CRD to configure the authentication by providing a set of Azure Active Directory credentials or by using pod identity. + +**Credential based authentication:** + +- `activeDirectoryClientId` - Id of the Active Directory application which requires at least `Monitoring Reader` permissions. +- `activeDirectoryClientPassword` - Password of the Active Directory application. + +The user will need access to read data from the Azure resource. + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: azure-monitor-secrets +data: + activeDirectoryClientId: + activeDirectoryClientPassword: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: azure-monitor-trigger-auth +spec: + secretTargetRef: + - parameter: activeDirectoryClientId + name: azure-monitor-secrets + key: activeDirectoryClientId + - parameter: activeDirectoryClientPassword + name: azure-monitor-secrets + key: activeDirectoryClientPassword + # or Pod Identity, kind: Secret is not required in case of pod Identity + podIdentity: + provider: azure +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: azure-monitor-scaler +spec: + scaleTargetRef: + name: azure-monitor-example + minReplicaCount: 1 + maxReplicaCount: 10 + triggers: + - type: azure-monitor + metadata: + resourceURI: Microsoft.ContainerService/managedClusters/azureMonitorCluster + tenantId: xxx-xxx-xxx-xxx-xxx + subscriptionId: yyy-yyy-yyy-yyy-yyy + resourceGroupName: azureMonitor + metricName: pod_custom_metric + metricNamespace: pod_custom_metrics_namespace # required when monitoring custom metrics + metricFilter: namespace eq 'default' + metricAggregationInterval: "0:1:0" + metricAggregationType: Average + targetValue: "1" + authenticationRef: + name: azure-monitor-trigger-auth +``` diff --git a/content/docs/2.7/scalers/azure-pipelines.md b/content/docs/2.7/scalers/azure-pipelines.md new file mode 100644 index 00000000000..2f15fc6e4dc --- /dev/null +++ b/content/docs/2.7/scalers/azure-pipelines.md @@ -0,0 +1,102 @@ ++++ +title = "Azure Pipelines" +layout = "scaler" +availability = "v2.3+" +maintainer = "Community" +description = "Scale applications based on agent pool queues for Azure Pipelines." +go_file = "azure_pipelines_scaler" ++++ + +### Trigger Specification + +This specification describes the `azure-pipelines` trigger for Azure Pipelines. It scales based on the amount of pipeline runs pending in a given agent pool. + +```yaml +triggers: + - type: azure-pipelines + metadata: + # Optional: Name of the pool in Azure DevOps + poolName: "{agentPoolName}" + # Optional: Learn more in 'How to determine your pool ID' + poolID: "{agentPoolId}" + # Optional: Azure DevOps organization URL, can use TriggerAuthentication as well + organizationURLFromEnv: "AZP_URL" + # Optional: Azure DevOps Personal Access Token, can use TriggerAuthentication as well + personalAccessTokenFromEnv: "AZP_TOKEN" + # Optional: Target queue length + targetPipelinesQueueLength: "1" # Default 1 + authenticationRef: + name: pipeline-trigger-auth +``` + +**Parameter list:** + +- `poolName` - Name of the pool. (Optional, either `poolID` or `poolName` must be configured) +- `poolID` - Id of the pool. (Optional, either `poolID` or `poolName` must be configured) +- `organizationURLFromEnv` - Name of the environment variable your deployment uses to get the URL for your Azure DevOps organization. +- `personalAccessTokenFromEnv` - Name of the environment variable that provides the personal access token (PAT) for Azure DevOps. Learn more about how to create one [in the official docs](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page). +- `targetPipelinesQueueLength` - Target value for the amount of pending jobs in the queue to scale on. (Default: `1`, Optional) + - Example - If one pod can handle 10 jobs, set the queue length target to 10. If the actual number of jobs in the queue is 30, the scaler scales to 3 pods. + +> 💡 **NOTE:** You can either use `poolID` or `poolName`. If both are specified, then `poolName` will be used. + +### Authentication Parameters + +As an alternative to using environment variables, you can authenticate with Azure Devops using a Personal Access Token via `TriggerAuthentication` configuration. + +**Personal Access Token Authentication:** + +- `organizationURL` - The URL of the Azure DevOps organization. +- `personalAccessToken` - The Personal Access Token (PAT) for Azure DevOps. + +### How to determine your pool ID + +There are several ways to get the `poolID`. The easiest could be using `az cli` to get it using the command `az pipelines pool list --pool-name {agentPoolName} --organization {organizationURL} --query [0].id`. + +It is also possible to get the pool ID using the UI by browsing to the agent pool from the organization (Organization settings -> Agent pools -> `{agentPoolName}`) and getting it from the URL. +The URL should be similar to `https://dev.azure.com/{organization}/_settings/agentpools?poolId={poolID}&view=jobs` + +> Careful - You should determine this on an organization-level, not project-level. Otherwise, you might get an incorrect id. + +Finally, it is also possible get the pool ID from the response of a HTTP request by calling the `https://dev.azure.com/{organizationName}/_apis/distributedtask/pools?poolname={agentPoolName}` endpoint in the key `value[0].id`. + +### Example + +```yaml +apiVersion: v1 +kind: Secret +type: Opaque +metadata: + name: pipeline-auth +data: + personalAccessToken: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: pipeline-trigger-auth + namespace: default +spec: + secretTargetRef: + - parameter: personalAccessToken + name: pipeline-auth + key: personalAccessToken +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: azure-pipelines-scaledobject + namespace: default +spec: + scaleTargetRef: + name: azdevops-deployment + minReplicaCount: 1 + maxReplicaCount: 5 + triggers: + - type: azure-pipelines + metadata: + poolID: "1" + organizationURLFromEnv: "AZP_URL" + authenticationRef: + name: pipeline-trigger-auth +``` diff --git a/content/docs/2.7/scalers/azure-service-bus.md b/content/docs/2.7/scalers/azure-service-bus.md new file mode 100644 index 00000000000..e2156806431 --- /dev/null +++ b/content/docs/2.7/scalers/azure-service-bus.md @@ -0,0 +1,93 @@ ++++ +title = "Azure Service Bus" +layout = "scaler" +maintainer = "Microsoft" +description = "Scale applications based on Azure Service Bus Queues or Topics." +availability = "v1.0+" +go_file = "azure_servicebus_scaler" ++++ + +### Trigger Specification + +This specification describes the `azure-servicebus` trigger for Azure Service Bus Queue or Topic. + +> ⚠️ **WARNING:** KEDA is not in charge of managing entities. If the queue, topic or subscription does not exist, it will not create them automatically. + +```yaml +triggers: +- type: azure-servicebus + metadata: + # Required: queueName OR topicName and subscriptionName + queueName: functions-sbqueue + # or + topicName: functions-sbtopic + subscriptionName: sbtopic-sub1 + # Optional, required when pod identity is used + namespace: service-bus-namespace + # Optional, can use TriggerAuthentication as well + connectionFromEnv: SERVICEBUS_CONNECTIONSTRING_ENV_NAME # This must be a connection string for a queue itself, and not a namespace level (e.g. RootAccessPolicy) connection string [#215](https://github.com/kedacore/keda/issues/215) + # Optional + messageCount: "5" # Optional. Count of messages to trigger scaling on. Default: 5 messages + cloud: Private # Optional. Default: AzurePublicCloud + endpointSuffix: servicebus.airgap.example # Required when cloud=Private +``` + +**Parameter list:** + +- `messageCount` - Amount of active messages in your Azure Service Bus queue or topic to scale on. +- `queueName` - Name of the Azure Service Bus queue to scale on. (Optional) +- `topicName` - Name of the Azure Service Bus topic to scale on. (Optional) +- `subscriptionName` - Name of the Azure Service Bus queue to scale on. (Optional*, Required when `topicName` is specified) +- `namespace` - Name of the Azure Service Bus namespace that contains your queue or topic. (Optional*, Required when pod identity is used) +- `connectionFromEnv` - Name of the environment variable your deployment uses to get the connection string of the Azure Service Bus namespace. (Optional) +- `cloud` - Name of the cloud environment that the service bus belongs to. Must be a known Azure cloud environment, or `Private` for Azure Stack Hub or Air Gapped clouds. (valid values: `AzurePublicCloud`, `AzureUSGovernmentCloud`, `AzureChinaCloud`, `AzureGermanCloud`, `Private`; default: `AzurePublicCloud`) + +When `cloud` is set to `Private`, the `endpointSuffix` parameter is required. Otherwise, it is automatically generated based on the cloud environment. `endpointSuffix` represents the service bus endpoint suffix of the cloud environment that the service bus belongs to, e.g. `servicebus.usgovcloudapi.net` for `AzureUSGovernmentCloud`. + +> 💡 **NOTE:** Service Bus Shared Access Policy needs to be of type `Manage`. Manage access is required for KEDA to be able to get metrics from Service Bus. + +### Authentication Parameters + +You can authenticate by using pod identity or connection string authentication. + +**Connection String Authentication:** + +- `connection` - Connection string for Azure Service Bus Namespace. + +### Example + +Here is an example of how to use managed identity: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: azure-servicebus-auth +spec: + podIdentity: + provider: azure +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: azure-servicebus-queue-scaledobject + namespace: default +spec: + scaleTargetRef: + name: azure-servicebus-queue-function + triggers: + - type: azure-servicebus + metadata: + # Required: queueName OR topicName and subscriptionName + queueName: functions-sbqueue + # or + topicName: functions-sbtopic + subscriptionName: sbtopic-sub1 + # Required: Define what Azure Service Bus to authenticate to with Managed Identity + namespace: service-bus-namespace + # Optional + messageCount: "5" # default 5 + cloud: AzureGermanCloud # Optional. Default: AzurePublicCloud + authenticationRef: + name: azure-servicebus-auth # authenticationRef would need either podIdentity or define a connection parameter +``` diff --git a/content/docs/2.7/scalers/azure-storage-blob.md b/content/docs/2.7/scalers/azure-storage-blob.md new file mode 100644 index 00000000000..0a710135314 --- /dev/null +++ b/content/docs/2.7/scalers/azure-storage-blob.md @@ -0,0 +1,93 @@ ++++ +title = "Azure Blob Storage" +layout = "scaler" +availability = "v1.1+" +maintainer = "Community" +description = "Scale applications based on the count of blobs in a given Azure Blob Storage container." +notice = "As of now, this Azure Blob Storage scaler scales based on the count of the blobs in a container as opposed to the Azure Functions behavior where code is only triggered on new blobs." +go_file = "azure_blob_scaler" ++++ + +### Trigger Specification + +This specification describes the `azure-blob` trigger for Azure Blob Storage. It scales based on the count of blobs in a given blob storage container and assumes the worker is responsible for clearing the container by deleting/moving the blobs once the blob processing completed. + +```yaml +triggers: +- type: azure-blob + metadata: + blobContainerName: functions-blob + blobCount: '5' + connectionFromEnv: STORAGE_CONNECTIONSTRING_ENV_NAME + accountName: storage-account-name + blobPrefix: myprefix + blobDelimiter: /example + cloud: Private + endpointSuffix: blob.core.airgap.example # Required when cloud=Private +``` + +**Parameter list:** + +- `blobContainerName` - Name of container in an Azure Storage account. +- `blobCount` - Average target value to trigger scaling actions. (Default: `5`, Optional) +- `connectionFromEnv` - Name of the environment variable your deployment uses to get the connection string. +- `accountName` - Name of the storage account that the container belongs to. +- `blobPrefix` - Prefix for the Blob. Use this to specify sub path for the blobs if required. (Default: `""`, Optional) +- `blobDelimiter` - Delimiter for identifying the blob prefix. (Default: `/`, Optional) +- `cloud` - Name of the cloud environment that the blob belongs to. Must be a known Azure cloud environment, or `Private` for Azure Stack Hub or Air Gapped clouds. (valid values: `AzurePublicCloud`, `AzureUSGovernmentCloud`, `AzureChinaCloud`, `AzureGermanCloud`, `Private`; default: `AzurePublicCloud`) + +When `cloud` is set to `Private`, the `endpointSuffix` parameter is required. Otherwise, it is automatically generated based on the cloud environment. `endpointSuffix` represents the storage blob endpoint suffix of the cloud environment that the blob belongs to, e.g. `blob.core.cloudapi.de` for `AzureGermanCloud`. + +You can also optionally assign a name to the metric using the `metricName` value. If not specified, the `metricName` will be generated automatically based on the trigger index and `blobContainerName`. For example: **s0-azure-blob-blobContainerName**. The value will be prefixed with `s{triggerIndex}-azure-blob-`. + +### Authentication Parameters + +You can authenticate by using pod identity or connection string authentication. + +**Connection String Authentication:** + +- `connection` - Connection string for Azure Storage Account. + +**Pod Identity Authentication** + +- `accountName` - Name of the Azure Storage Account. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: azure-blob-auth +spec: + podIdentity: + provider: azure +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: azure-blob-scaledobject + namespace: default +spec: + scaleTargetRef: + name: azureblob-function + triggers: + - type: azure-blob + metadata: + # Required + blobContainerName: functionsblob + # Optional, required when pod identity is used + accountName: storage-account-name + # Optional, connection OR authenticationRef that defines the connection + connectionFromEnv: STORAGE_CONNECTIONSTRING_ENV_NAME # Reference to a connection string in deployment + # or authenticationRef as defined below + # + # Optional + blobCount: "5" # default 5 + blobPrefix: blobsubpath # Default : "" + blobDelimiter: "/" # Default: "/" + # Optional, default: AzurePublicCloud + cloud: AzureChinaCloud + authenticationRef: + name: azure-blob-auth # authenticationRef would need either podIdentity or define a connection parameter +``` diff --git a/content/docs/2.7/scalers/azure-storage-queue.md b/content/docs/2.7/scalers/azure-storage-queue.md new file mode 100644 index 00000000000..b9e0368e2ba --- /dev/null +++ b/content/docs/2.7/scalers/azure-storage-queue.md @@ -0,0 +1,79 @@ ++++ +title = "Azure Storage Queue" +layout = "scaler" +availability = "v1.0+" +maintainer = "Microsoft" +description = "Scale applications based on Azure Storage Queues." +go_file = "azure_queue_scaler" ++++ + +### Trigger Specification + +This specification describes the `azure-queue` trigger for Azure Storage Queue. + +```yaml +triggers: +- type: azure-queue + metadata: + queueName: orders + queueLength: '5' + connectionFromEnv: STORAGE_CONNECTIONSTRING_ENV_NAME + accountName: storage-account-name + cloud: AzureUSGovernmentCloud +``` + +**Parameter list:** + +- `queueName` - Name of the queue. +- `queueLength` - Target value for queue length passed to the scaler. Example: if one pod can handle 10 messages, set the queue length target to 10. If the actual number of messages in the queue is 30, the scaler scales to 3 pods. (Default: `5`, Optional) +- `connectionFromEnv` - Name of the environment variable your deployment uses to get the connection string. +- `accountName` - Name of the storage account that the queue belongs to. +- `cloud` - Name of the cloud environment that the queue belongs to. Must be a known Azure cloud environment, or `Private` for Azure Stack Hub or Air Gapped clouds. (valid values: `AzurePublicCloud`, `AzureUSGovernmentCloud`, `AzureChinaCloud`, `AzureGermanCloud`, `Private`; default: `AzurePublicCloud`) + +When `cloud` is set to `Private`, the `endpointSuffix` parameter is required. Otherwise, it is automatically generated based on the cloud environment. `endpointSuffix` represents the storage queue endpoint suffix of the cloud environment that the queue belongs to, e.g. `queue.core.windows.net` for `AzurePublicCloud`. + +### Authentication Parameters + +You can authenticate by using pod identity or connection string authentication. + +**Connection String Authentication:** + +- `connection` - Connection string for Azure Storage Account. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: azure-queue-auth +spec: + podIdentity: + provider: azure +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: azure-queue-scaledobject + namespace: default +spec: + scaleTargetRef: + name: azurequeue-function + triggers: + - type: azure-queue + metadata: + # Required + queueName: functionsqueue + # Optional, required when pod identity is used + accountName: storage-account-name + # Optional: connection OR authenticationRef that defines connection + connectionFromEnv: STORAGE_CONNECTIONSTRING_ENV_NAME # Default: AzureWebJobsStorage. Reference to a connection string in deployment + # or authenticationRef as defined below + # + # Optional + queueLength: "5" # default 5 + cloud: Private + endpointSuffix: queue.local.azurestack.external # Required when cloud=Private + authenticationRef: + name: azure-queue-auth # authenticationRef would need either podIdentity or define a connection parameter +``` diff --git a/content/docs/2.7/scalers/cassandra.md b/content/docs/2.7/scalers/cassandra.md new file mode 100644 index 00000000000..e8c8a47ced1 --- /dev/null +++ b/content/docs/2.7/scalers/cassandra.md @@ -0,0 +1,90 @@ ++++ +title = "Cassandra" +availability = "v2.5+" +maintainer = "Community" +description = "Scale applications based on Cassandra query results." +layout = "scaler" +go_file = "cassandra_scaler" ++++ + +### Trigger Specification + +This specification describes the `cassandra` trigger that scales based on the outputs of a Cassandra query. + +```yaml +triggers: + - type: cassandra + metadata: + username: "cassandra" + port: "9042" + clusterIPAddress: "cassandra.default" + consistency: "Quorum" + protocolVersion: "4" + keyspace: "test_keyspace" + query: "SELECT COUNT(*) FROM test_keyspace.test_table;" + targetQueryValue: "1" + metricName: "test_keyspace" +``` + +**Parameter list:** + +- `username` - The username credential for connecting to the Cassandra instance. +- `port` - The port number of the Cassandra instance. (Optional, Can be set either here or in `clusterIPAddress`) +- `clusterIPAddress` - The IP address or the host name of the Cassandra instance. +- `consistency` - Configuration for a session or per individual read operation. (Values: `LOCAL_ONE`, `LOCAL_QUORUM`, `EACH_QUORUM`, `LOCAL_SERIAL`, `ONE`, `TWO`, `THREE`, `QUORUM`, `SERIAL`, `ALL`, Default: `ONE`, Optional) +- `protocolVersion` - CQL Binary Protocol. (Default: `4`, Optional) +- `keyspace` - The name of the keyspace used in Cassandra. +- `query` - A Cassandra query that should return single numeric value. +- `targetQueryValue` - The threshold value that is provided by the user and used as `targetAverageValue` in the Horizontal Pod Autoscaler (HPA). +- `metricName` - Name to assign to the metric. (Default: `s-cassandra-`, Optional, In case of `metricName` is specified, it will be used to generate the `metricName` like this: `s-cassandra-`, where `` is the index of the trigger in a ScaledObject) + +### Authentication Parameters + +You can authenticate by using a password via `TriggerAuthentication` configuration. + +**Password Authentication:** + +- `password` - Password for configured user to login to the Cassandra instance. + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: cassandra-secrets +type: Opaque +data: + cassandra_password: CASSANDRA_PASSWORD +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-cassandra-secret +spec: + secretTargetRef: + - parameter: password + name: cassandra-secrets + key: cassandra_password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: cassandra-scaledobject +spec: + scaleTargetRef: + name: nginx-deployment + triggers: + - type: cassandra + metadata: + username: "cassandra" + port: "9042" + clusterIPAddress: "cassandra.default" + consistency: "Quorum" + protocolVersion: "4" + query: "SELECT COUNT(*) FROM test_keyspace.test_table;" + targetQueryValue: "1" + metricName: "test_keyspace" + authenticationRef: + name: keda-trigger-auth-cassandra-secret +``` diff --git a/content/docs/2.7/scalers/cpu.md b/content/docs/2.7/scalers/cpu.md new file mode 100644 index 00000000000..b7d944fe0f4 --- /dev/null +++ b/content/docs/2.7/scalers/cpu.md @@ -0,0 +1,50 @@ ++++ +title = "CPU" +layout = "scaler" +availability = "v2.0+" +maintainer = "Community" +description = "Scale applications based on cpu metrics." +go_file = "cpu_memory_scaler" ++++ + +> **Notice:** +> - This scaler will never scale to 0 and even when user defines multiple scaler types (eg. Kafka + cpu/memory, or Prometheus + cpu/memory), the deployment will never scale to 0. +> - This scaler only applies to ScaledObject, not to Scaling Jobs. + +### Trigger Specification + +This specification describes the `cpu` trigger that scales based on cpu metrics. + +```yaml +triggers: +- type: cpu + metadata: + # Required + type: Utilization/ AverageValue + value: "60" +``` + +**Parameter list:** + +- `type` - Type of metric to use. Options are `Utilization`, or `AverageValue`. +- `value` - Value to trigger scaling actions for: + - When using `Utilization`, the target value is the average of the resource metric across all relevant pods, represented as a percentage of the requested value of the resource for the pods. + - When using `AverageValue`, the target value is the target value of the average of the metric across all relevant pods (quantity). + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: cpu-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + triggers: + - type: cpu + metadata: + type: Utilization + value: "50" +``` diff --git a/content/docs/2.7/scalers/cron.md b/content/docs/2.7/scalers/cron.md new file mode 100644 index 00000000000..df767e60c76 --- /dev/null +++ b/content/docs/2.7/scalers/cron.md @@ -0,0 +1,61 @@ ++++ +title = "Cron" +layout = "scaler" +availability = "v1.5+" +maintainer = "Community" +description = "Scale applications based on a cron schedule." +go_file = "cron_scaler" ++++ + +### Trigger Specification + +This specification describes the `cron` trigger that scales based on a Cron Schedule. + +```yaml +triggers: +- type: cron + metadata: + # Required + timezone: Asia/Kolkata # The acceptable values would be a value from the IANA Time Zone Database. + start: 30 * * * * # Every hour on the 30th minute + end: 45 * * * * # Every hour on the 45th minute + desiredReplicas: "10" +``` + +**Parameter list:** + +- `timezone` - One of the acceptable values from the IANA Time Zone Database. The list of timezones can be found [here](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). +- `start` - Cron expression indicating the start of the cron schedule. +- `end` - Cron expression indicating the end of the cron schedule. +- `desiredReplicas` - Number of replicas to which the resource has to be scaled **between the start and end** of the cron schedule. + +> 💡 **Note:** `start`/`end` support ["Linux format cron"](https://en.wikipedia.org/wiki/Cron) (Minute Hour Dom Month Dow). + +> **Notice:** +> **Start and end should not be same.** +> +> For example, the following schedule is not valid: +> ```yaml +> start: 30 * * * * +> end: 30 * * * * +>``` + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: cron-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + triggers: + - type: cron + metadata: + timezone: Asia/Kolkata + start: 30 * * * * + end: 45 * * * * + desiredReplicas: "10" +``` diff --git a/content/docs/2.7/scalers/datadog.md b/content/docs/2.7/scalers/datadog.md new file mode 100644 index 00000000000..8143346392e --- /dev/null +++ b/content/docs/2.7/scalers/datadog.md @@ -0,0 +1,97 @@ ++++ +title = "Datadog" +layout = "scaler" +availability = "v2.6+" +maintainer = "Datadog" +description = "Scale applications based on Datadog." +go_file = "datadog_scaler" ++++ + +### Trigger Specification + +This specification describes the `datadog` trigger that scales based on a Datadog metric. + +```yaml +triggers: +- type: datadog + metadata: + query: "sum:trace.redis.command.hits{env:none,service:redis}.as_count()" + queryValue: "7" + type: "global" + age: "60" +``` + +**Parameter list:** + +- `query` - The Datadog query to run. +- `queryValue` - Value to reach to start scaling. +- `type` - Whether to start scaling based on the value or the average between pods. (Values: `average`, `global`, Default:`average`, Optional) +- `age`: The time window (in seconds) to retrieve metrics from Datadog. (Default: `90`, Optional) + +### Authentication + +Datadog requires both an API key and an APP key to retrieve metrics from your account. + +You should use `TriggerAuthentication` CRD to configure the authentication: + +**Parameter list:** +- `apiKey` - Datadog API key. +- `appKey` - Datadog APP key. +- `datadogSite` - Datadog site where to get the metrics from. This is commonly referred as DD_SITE in Datadog documentation. (Default: `datadoghq.com`, Optional) + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: datadog-secrets + namespace: my-project +type: Opaque +data: + apiKey: # Required: base64 encoded value of Datadog apiKey + appKey: # Required: base64 encoded value of Datadog appKey + datadogSite: # Optional: base64 encoded value of Datadog site +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-datadog-secret + namespace: my-project +spec: + secretTargetRef: + # Required: API key for your Datadog account + - parameter: apiKey + name: datadog-secrets + key: apiKey + # Required: APP key for your Datadog account + - parameter: appKey + name: datadog-secrets + key: appKey + # Optional: Datadog site. Default: "datadoghq.com" + - parameter: datadogSite + name: datadog-secrets + key: datadogSite +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: datadog-scaledobject + namespace: my-project +spec: + scaleTargetRef: + name: worker + triggers: + - type: datadog + metadata: + # Required: datadog metric query + query: "sum:trace.redis.command.hits{env:none,service:redis}.as_count()" + # Required: according to the number of query result, to scale the TargetRef + queryValue: "7" + # Optional: (Global or Average). Whether the target value is global or average per pod. Default: Average + type: "Global" + # Optional: The time window (in seconds) to retrieve metrics from Datadog. Default: 90 + age: "60" + authenticationRef: + name: keda-trigger-auth-datadog-secret +``` diff --git a/content/docs/2.7/scalers/elasticsearch.md b/content/docs/2.7/scalers/elasticsearch.md new file mode 100644 index 00000000000..11d7956bd52 --- /dev/null +++ b/content/docs/2.7/scalers/elasticsearch.md @@ -0,0 +1,93 @@ ++++ +title = "Elasticsearch" +availability = "v2.5+" +maintainer = "Community" +description = "Scale applications based on elasticsearch search template query result." +layout = "scaler" +go_file = "elasticsearch_scaler" ++++ + +### Trigger Specification + +This specification describes the `elasticsearch` trigger that scales based on result of an [elasticsearch search template](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-template.html) query. + +The trigger always requires the following information: + +```yaml +triggers: + - type: elasticsearch + metadata: + addresses: "http://localhost:9200" + username: "elastic" + passwordFromEnv: "ELASTIC_PASSWORD" + index: "my-index" + searchTemplateName: "my-search-template-name" + params: "param1:value1;param2:value2" + valueLocation: "hits.total.value" + targetValue: "1" +``` + +**Parameter list:** + +- `addresses` - Comma separated list of hosts and ports of the Elasticsearch cluster client nodes. +- `username` - Username to authenticate with to Elasticsearch cluster. +- `passwordFromEnv` - Environment variable to read the authentication password from to authenticate with the Elasticsearch cluster. +- `index` - Comma separated list of indexes to run the search template query on. +- `searchTemplateName` - The search template name to run. +- `targetValue` - Target value to scale on. When the metric provided by the API is equal or higher to this value, KEDA will start scaling out. When the metric is 0 or less, KEDA will scale down to 0. +- `parameters` - Parameters that will be used by the search template. It supports multiples params separated by a semicolon character ( `;` ). +- `valueLocation` - [GJSON path notation](https://github.com/tidwall/gjson#path-syntax) to refer to the field in the payload containing the metric value. +- `unsafeSsl` - Skip certificate validation when connecting over HTTPS. (Values: `true`, `false`, Default: `false`, Optional) + +### Authentication Parameters + +You can authenticate by using a username/password authentication. + +**Password Authentication:** + +- `username` - Username to authenticate with to Elasticsearch cluster. +- `password` - Password for configured user to login to Elasticsearch cluster. + +### Example + +Here is an example of how to deploy a scaled object with the `elasticsearch` scale trigger which uses `TriggerAuthentication`. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: elasticsearch-secrets +type: Opaque +data: + password: cGFzc3cwcmQh +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-elasticsearch-secret +spec: + secretTargetRef: + - parameter: password + name: elasticsearch-secrets + key: password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: elasticsearch-scaledobject +spec: + scaleTargetRef: + name: "deployment-name" + triggers: + - type: elasticsearch + metadata: + addresses: "http://localhost:9200" + username: "elastic" + index: "my-index" + searchTemplateName: "my-search-template" + valueLocation: "hits.total.value" + targetValue: "10" + params: "dummy_value:1" + authenticationRef: + name: keda-trigger-auth-elasticsearch-secret +``` diff --git a/content/docs/2.7/scalers/external-push.md b/content/docs/2.7/scalers/external-push.md new file mode 100644 index 00000000000..f2c6a6c52f3 --- /dev/null +++ b/content/docs/2.7/scalers/external-push.md @@ -0,0 +1,51 @@ ++++ +title = "External Push" +layout = "scaler" +availability = "v2.0+" +maintainer = "Microsoft" +description = "Scale applications based on an external push scaler." +go_file = "external_scaler" ++++ + +### Trigger Specification + +This specification describes the `external-push` trigger for an external push scaler. + +```yaml +triggers: +- type: external-push + metadata: + scalerAddress: external-scaler-service:8080 + tlsCertFile: /path/to/tls/cert.pem # optional +``` + +**Parameter list:** + +- `scalerAddress` - Address of the external push scaler implementing `ExternalScaler.StreamIsActive` in externalscaler.proto. Format must be `host:port`. +- `tlsCertFile` - Location of a certificate to use for the GRPC connection to authenticate with. (Optional) + +The entire metadata object is passed to the external scaler in `ScaledObjectRef.scalerMetadata`. + +> For implementing an external scaler, refer to [External Scalers Concept](../concepts/external-scalers.md). + +### Authentication Parameters + +Not supported. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: name + namespace: namespace +spec: + scaleTargetRef: + name: keda-node + triggers: + - type: external-push + metadata: + scalerAddress: external-scaler-service:8080 + tlsCertFile: /path/to/tls/cert.pem # optional +``` diff --git a/content/docs/2.7/scalers/external.md b/content/docs/2.7/scalers/external.md new file mode 100644 index 00000000000..293114eff9d --- /dev/null +++ b/content/docs/2.7/scalers/external.md @@ -0,0 +1,52 @@ ++++ +title = "External" +layout = "scaler" +availability = "v1.0+" +maintainer = "Microsoft" +description = "Scale applications based on an external scaler." +go_file = "external_scaler" ++++ + +### Trigger Specification + +This specification describes the `external` trigger for an external scaler. + +```yaml +triggers: +- type: external + metadata: + scalerAddress: external-scaler-service:8080 + tlsCertFile : /path/to/tls/cert.pem +``` + +**Parameter list:** + +- `scalerAddress` - Address of the external scaler. Format must be `host:port`. +- `tlsCertFile` - Location of a certificate to use for the GRPC connection to authenticate with. (Optional) + +> For implementing an external scaler, refer to [External Scalers Concept](../concepts/external-scalers.md). + +### Authentication Parameters + +Not supported yet. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-scaledobject + namespace: keda-redis-test +spec: + scaleTargetRef: + name: keda-redis-node + triggers: + - type: external + metadata: + scalerAddress: redis-external-scaler-service:8080 + address: REDIS_HOST + password: REDIS_PASSWORD + listName: mylist + listLength: "5" +``` diff --git a/content/docs/2.7/scalers/gcp-pub-sub.md b/content/docs/2.7/scalers/gcp-pub-sub.md new file mode 100644 index 00000000000..004f6770ca6 --- /dev/null +++ b/content/docs/2.7/scalers/gcp-pub-sub.md @@ -0,0 +1,151 @@ ++++ +title = "Google Cloud Platform‎ Pub/Sub" +layout = "scaler" +availability = "v1.0+" +maintainer = "Community" +description = "Scale applications based on Google Cloud Platform‎ Pub/Sub." +go_file = "gcp_pub_sub_scaler" ++++ + +### Trigger Specification + +This specification describes the `gcp-pubsub` trigger for Google Cloud Platform‎ Pub/Sub. + +```yaml +triggers: +- type: gcp-pubsub + metadata: + subscriptionSize: "5" # Deprecated, use mode and value fields instead + mode: "SubscriptionSize" # Optional - Default is SubscriptionSize - SubscriptionSize or OldestUnackedMessageAge + value: "5" # Optional - Default is 5 for SubscriptionSize | Default is 10 for OldestUnackedMessageAge + subscriptionName: "mysubscription" # Required + credentialsFromEnv: GOOGLE_APPLICATION_CREDENTIALS_JSON # Required +``` + +The Google Cloud Platform‎ (GCP) Pub/Sub trigger allows you to scale based on the number of messages or oldest unacked message age in your Pub/Sub subscription. + +The `credentialsFromEnv` property maps to the name of an environment variable in the scale target (`scaleTargetRef`) that contains the service account credentials (JSON). KEDA will use those to connect to Google Cloud Platform and collect the required stack driver metrics in order to read the number of messages in the Pub/Sub subscription. + +`subscriptionName` defines the subscription that should be monitored. You can use different formulas: + +- Just the subscription name, in which case you will reference a subscription from the current project or the one specified in the credentials file used. +- Use the full link provided by Google, so that you can reference a subscription that is hosted in another project Eg: `projects/myproject/subscriptions/mysubscription`. + +You can use either `subscriptionSize` to define the target average which the deployment will be scaled on or `mode` and `value` fields. `subscriptionSize` field is deprecated, it is recommended to use `mode` and `value` fields instead. Scaler will not work if you define both `subscriptionSize` and at least one of `mode` or `value`. +The mode chooses whether to scale using number of messages `SubscriptionSize` or using oldest unacked message age `OldestUnackedMessageAge`. +The `value` determines the target average which the deployment will be scaled on. The default value is 5 for `SubscriptionSize` and 10 for `OldestUnackedMessageAge`. + +Here's an [example](https://github.com/kedacore/sample-go-gcppubsub). + +### Authentication Parameters +You can use `TriggerAuthentication` CRD to configure the authenticate by providing the service account credentials in JSON. + + +**Credential based authentication:** + +- `GoogleApplicationCredentials` - Service account credentials in JSON. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: pubsub-scaledobject + namespace: keda-pubsub-test +spec: + scaleTargetRef: + name: keda-pubsub-go + triggers: + - type: gcp-pubsub + metadata: + mode: "SubscriptionSize" + value: "5" + subscriptionName: "mysubscription" # Required + credentialsFromEnv: GOOGLE_APPLICATION_CREDENTIALS_JSON # Required +``` + +### Example using TriggerAuthentication + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-gcp-credentials +spec: + secretTargetRef: + - parameter: GoogleApplicationCredentials + name: pubsub-secret # Required. Refers to the name of the secret + key: GOOGLE_APPLICATION_CREDENTIALS_JSON # Required. +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: pubsub-scaledobject +spec: + scaleTargetRef: + name: keda-pubsub-go + triggers: + - type: gcp-pubsub + authenticationRef: + name: keda-trigger-auth-gcp-credentials + metadata: + subscriptionName: "input" # Required +``` + +**Identity based authentication:** + +You can also use `TriggerAuthentication` CRD to configure the authentication using the associated service account of the running machine in Google Cloud. You only need to create a `TriggerAuthentication` as this example, and reference it in the `ScaledObject`. `ClusterTriggerAuthentication` can also be used if you pretend to use it globally in your cluster. + +### Example using TriggerAuthentication with GCP Identity + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-gcp-credentials +spec: + podIdentity: + provider: gcp +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: pubsub-scaledobject +spec: + scaleTargetRef: + name: keda-pubsub-go + triggers: + - type: gcp-pubsub + authenticationRef: + name: keda-trigger-auth-gcp-credentials + metadata: + subscriptionName: "input" # Required +``` + +## Example using ClusterTriggerAuthentication with GCP Identity + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ClusterTriggerAuthentication +metadata: + name: keda-clustertrigger-auth-gcp-credentials +spec: + podIdentity: + provider: gcp +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: pubsub-scaledobject +spec: + scaleTargetRef: + name: keda-pubsub-go + triggers: + - type: gcp-pubsub + authenticationRef: + name: keda-clustertrigger-auth-gcp-credentials + kind: ClusterTriggerAuthentication + metadata: + subscriptionName: "input" # Required +``` \ No newline at end of file diff --git a/content/docs/2.7/scalers/graphite.md b/content/docs/2.7/scalers/graphite.md new file mode 100644 index 00000000000..a65c06db6f2 --- /dev/null +++ b/content/docs/2.7/scalers/graphite.md @@ -0,0 +1,114 @@ ++++ +title = "Graphite" +layout = "scaler" +availability = "v2.5+" +maintainer = "Community" +description = "Scale applications based on metrics in Graphite." +go_file = "graphite_scaler" ++++ + +### Trigger Specification + +This specification describes the `graphite` trigger that scales based on metrics in Graphite. + +```yaml +triggers: +- type: graphite + metadata: + # Required + serverAddress: http://:81 + metricName: request-count # Note: name to identify the metric + query: stats.counters.http.hello-world.request.count.count # Note: query must return a vector/scalar single element response + threshold: '100' + queryTime: '-10Minutes' # Note: Query time in from argv Seconds/Minutes/Hours +``` +**Parameter list:** + +- `serverAddress` - Address of Graphite +- `metricName` - Name to identify the Metric in the external.metrics.k8s.io API. If using more than one trigger it is required that all `metricName`(s) be unique +- `query` - Query to run. +- `threshold` - Value to start scaling for. +- `queryTime` - Relative time range to execute query against. Please see the [graphite API docs](https://graphite-api.readthedocs.io/en/latest/api.html#from-until) for more information. + +### Authentication Parameters + +Graphite Scaler supports one type of authentication - basic authentication + +**Basic authentication:** +- `authMode`: It must contain `basic` in case of Basic Authentication. Specify this in trigger configuration. +- `username` - This is a required field. Provide the username to be used for basic authentication. +- `password` - Provide the password to be used for authentication. For convenience, this has been marked optional, because many applications implement basic auth with a username as apikey and password as empty. + +### Examples + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: graphite-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + triggers: + - type: graphite + metadata: + serverAddress: http://:81 + metricName: LagMetric + threshold: '100' + query: maxSeries(keepLastValue(reportd.*.gauge.detect.latest_max_time.value, 1)) + queryTime: '-1Minutes' +``` + +Here is an example of a Graphite Scaler with Basic Authentication, define the `Secret` and `TriggerAuthentication` as follows + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-graphite-secret + namespace: default +data: + username: "dXNlcm5hbWUK" # Must be base64 + password: "cGFzc3dvcmQK" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-graphite-creds + namespace: default +spec: + secretTargetRef: + - parameter: username + name: keda-graphite-secret + key: username + - parameter: password + name: keda-graphite-secret + key: password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: graphite-scaledobject + namespace: default + labels: + deploymentName: php-apache-graphite +spec: + cooldownPeriod: 10 + maxReplicaCount: 5 + minReplicaCount: 0 + pollingInterval: 5 + scaleTargetRef: + name: php-apache-graphite + triggers: + - type: graphite + metadata: + authMode: "basic" + metricName: https_metric + query: https_metric + queryTime: -1Hours + serverAddress: http://:81 + threshold: "100" + authenticationRef: + name: keda-graphite-creds +``` diff --git a/content/docs/2.7/scalers/huawei-cloudeye.md b/content/docs/2.7/scalers/huawei-cloudeye.md new file mode 100644 index 00000000000..779684a5cc3 --- /dev/null +++ b/content/docs/2.7/scalers/huawei-cloudeye.md @@ -0,0 +1,122 @@ ++++ +title = "Huawei Cloudeye" +layout = "scaler" +availability = "v1.1+" +maintainer = "Community" +description = "Scale applications based on a Huawei Cloudeye." +go_file = "huawei_cloudeye_scaler" ++++ + +### Trigger Specification + +This specification describes the `huawei-cloudeye` trigger that scales based on a Huawei Cloudeye. + +```yaml +triggers: +- type: huawei-cloudeye + metadata: + namespace: SYS.ELB + metricName: mb_l7_qps + dimensionName: lbaas_instance_id + dimensionValue: 5e052238-0346-xxb0-86ea-92d9f33e29d2 + targetMetricValue: "100" + minMetricValue: "1" +``` + +**Parameter list:** + +- `namespace` - Namespace of the metric.The format is service.item; service and item must be strings, must start with a letter, can only contain 0-9/a-z/A-Z/_, the total length of service.item is 3, the maximum is 32. +- `metricName` - Name of the metric. +- `dimensionName` - Name of the metric dimension. +- `dimensionValue` - Value of the metric dimension. +- `targetMetricValue` - Target value for your metric. +- `minMetricValue` - Minimum value for your metric. If the actual value of the metric you get from cloudeye is less than the minimum value, then the scaler is not active. +- `metricCollectionTime` - Collection time of the metric in seconds. Equivalent to the earliest start time of the end time. (default: 300) +- `metricFilter` - Aggregation method of the metric. (Values: `average`, `max`, `min`, `sum`, Default: `average`, Optional) +- `metricPeriod` - Granularity of the metric in seconds. (Default: 300, Optional) + +### Authentication Parameters + +You can use `TriggerAuthentication` CRD to configure the authenticate by providing a set of IAM credentials. + +**Credential based authentication:** + +- `IdentityEndpoint` - Identity Endpoint. +- `ProjectID` - Project ID. +- `DomainID` - Id of domain. +- `Domain` - Domain. +- `Region` - Region. +- `Cloud` - Cloud name. (Default: `myhuaweicloud.com`, Optional) +- `AccessKey` - Id of the user. +- `SecretKey` - Access key for the user to authenticate with. + +The user will need access to read data from Huawei Cloudeye. + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-huawei-secrets + namespace: keda-test +data: + IdentityEndpoint: + ProjectID: + DomainID: + Region: + Domain: + AccessKey: + SecretKey: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-huawei-credential + namespace: keda-test +spec: + secretTargetRef: + - parameter: IdentityEndpoint # Required. + name: keda-huawei-secrets # Required. + key: IdentityEndpoint # Required. + - parameter: ProjectID # Required. + name: keda-huawei-secrets # Required. + key: ProjectID # Required. + - parameter: DomainID # Required. + name: keda-huawei-secrets # Required. + key: DomainID # Required. + - parameter: Region # Required. + name: keda-huawei-secrets # Required. + key: Region # Required. + - parameter: Domain # Required. + name: keda-huawei-secrets # Required. + key: Domain # Required. + - parameter: AccessKey # Required. + name: keda-huawei-secrets # Required. + key: AccessKey # Required. + - parameter: SecretKey # Required. + name: keda-huawei-secrets # Required. + key: SecretKey # Required. +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: huawei-cloudeye-scaledobject + namespace: keda-test +spec: + scaleTargetRef: + name: nginx-deployment + maxReplicaCount: 5 + minReplicaCount: 2 + triggers: + - type: huawei-cloudeye + metadata: + namespace: SYS.ELB + dimensionName: lbaas_instance_id + dimensionValue: 5e052238-0346-47b0-xxea-92d9f33e29d2 + metricName: mb_l7_qps + targetMetricValue: "100" + minMetricValue: "1" + authenticationRef: + name: keda-trigger-auth-huawei-credential +``` diff --git a/content/docs/2.7/scalers/ibm-mq.md b/content/docs/2.7/scalers/ibm-mq.md new file mode 100644 index 00000000000..a1bbebb713f --- /dev/null +++ b/content/docs/2.7/scalers/ibm-mq.md @@ -0,0 +1,98 @@ ++++ +title = "IBM MQ" +layout = "scaler" +availability = "v2.0+" +maintainer = "Community" +description = "Scale applications based on IBM MQ Queue" +go_file = "ibmmq_scaler" ++++ + +### Trigger Specification + +This specification describes the `ibmmq` trigger for IBM MQ Queue. + +```yaml +triggers: +- type: ibmmq + metadata: + host: # REQUIRED - IBM MQ Queue Manager Admin REST Endpoint + queueManager: # REQUIRED - Your queue manager + queueName: # REQUIRED - Your queue name + tlsDisabled: # OPTIONAL - Set 'true' to disable TLS. Default: false + queueDepth: # OPTIONAL - Queue depth target for HPA. Default: 5 messages + usernameFromEnv: # Optional: Provide admin username from env instead of as a secret + passwordFromEnv: # Optional: Provide admin password from env instead of as a secret +``` + +**Parameter list:** + +- `host` - IBM MQ Queue Manager Admin REST Endpoint. Example URI endpoint structure on IBM cloud `https://example.mq.appdomain.cloud/ibmmq/rest/v2/admin/action/qmgr/QM/mqsc`. +- `queueManager` - Name of the queue manager from which messages will be consumed. +- `queueName` - Name of the Queue within the Queue Manager defined from which messages will be consumed. +- `tlsDisabled` - Can be set to 'true' to disable TLS. (Values: `true`, `false` , Default: `false`, Optional) +- `queueDepth` - Queue depth Target for HPA. (Default: `5`, Optional) +- `usernameFromEnv` - Provide admin username from env instead of as a secret. (Optional) +- `passwordFromEnv` - Provide admin password from env instead of as a secret. (Optional) + +### Authentication Parameters + +TriggerAuthentication CRD is used to connect and authenticate to IBM MQ: + +**Authentication Parameters** + +- `ADMIN_USER` - REQUIRED - The admin REST endpoint username for your MQ Queue Manager`. +- `ADMIN_PASSWORD` - REQUIRED - The admin REST endpoint API key for your MQ Queue Manager. +- `usernameFromEnv` - Provide admin username from env instead of as a secret. (Optional) +- `passwordFromEnv` - Provide admin password from env instead of as a secret. (Optional) + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-ibmmq-secret +data: + ADMIN_USER: # REQUIRED - Admin Username + ADMIN_PASSWORD: # REQUIRED - Admin Password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: ibmmq-scaledobject + namespace: default + labels: + deploymentName: ibmmq-deployment +spec: + scaleTargetRef: + name: ibmmq-deployment + pollingInterval: 5 # OPTIONAL - Default: 30 seconds + cooldownPeriod: 30 # OPTIONAL - Default: 300 seconds + maxReplicaCount: 18 # OPTIONAL - Default: 100 + triggers: + - type: ibmmq + metadata: + host: # REQUIRED - IBM MQ Queue Manager Admin REST Endpoint + queueManager: # REQUIRED - Your queue manager + queueName: # REQUIRED - Your queue name + tlsDisabled: # OPTIONAL - Set 'true' to disable TLS. Default: false + queueDepth: # OPTIONAL - Queue depth target for HPA. Default: 5 messages + usernameFromEnv: # Optional: Provide admin username from env instead of as a secret + passwordFromEnv: # Optional: Provide admin password from env instead of as a secret + authenticationRef: + name: keda-ibmmq-trigger-auth +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-ibmmq-trigger-auth + namespace: default +spec: + secretTargetRef: + - parameter: username + name: keda-ibmmq-secret + key: ADMIN_USER + - parameter: password + name: keda-ibmmq-secret + key: ADMIN_PASSWORD +``` diff --git a/content/docs/2.7/scalers/influxdb.md b/content/docs/2.7/scalers/influxdb.md new file mode 100644 index 00000000000..c9ded5d6dd7 --- /dev/null +++ b/content/docs/2.7/scalers/influxdb.md @@ -0,0 +1,75 @@ ++++ +title = "InfluxDB" +layout = "scaler" +availability = "v2.1+" +maintainer = "Community" +description = "Scale applications based on InfluxDB queries" +go_file = "influxdb_scaler" ++++ + +### Trigger Specification + +This specification describes the `influxdb` trigger that scales based on the results of a InfluxDB query. + +```yaml +triggers: + - type: influxdb + metadata: + serverURL: http://influxdb:8086 + organizationName: influx-org + organizationNameFromEnv: INFLUXDB_ORG_NAME # Optional: You can use this instead of `organizationName` parameter. See details in "Parameter List" section + thresholdValue: '4' + query: | + from(bucket: "bucket-of-interest") + |> range(start: -12h) + |> filter(fn: (r) => r._measurement == "stat") + authToken: some-auth-token + authTokenFromEnv: INFLUXDB_AUTH_TOKEN # Optional: You can use this instead of `authToken` parameter. See details in "Parameter List" section + metricName: influx-metric # Optional: This value will default to organization name if not set by the user (metrics name value would be then `influxdb-influx_org`) +``` + +**Parameter list:** + +- `authToken` - Authentication token needed for the InfluxDB client to communicate with an associated server. +- `authTokenFromEnv` - Defines the authorization token, similar to `authToken`, but reads it from an environment variable on the scale target. +- `organizationName` - Organization name needed for the client to locate all information contained in that [organization](https://docs.influxdata.com/influxdb/v2.0/organizations/) such as buckets, tasks, etc. +- `organizationNameFromEnv` - Defines the organization name, similar to `organizationName`, but reads it from an environment variable on the scale target. +- `serverURL` - Holds the url value of the InfluxDB server. +- `thresholdValue` - Provided by the user. This value can vary from use case to use case depending on the data of interest, and is needed to trigger the scaling in/out depending on what value comes back from the query. +- `query` - Flux query that will yield the value for the scaler to compare the `thresholdValue` against. +- `metricName` - Name to assign to the metric. If not set KEDA will generate a name based organization name. If using more than one trigger it is required that all `metricName`(s) be unique. (Optional) +- `unsafeSsl` - Skip certificate validation when connecting over HTTPS. (Values: `true`, `false`, Default: `false`, Optional) + +### Authentication Parameters + +You can authenticate by using an authorization token. + +**Authorization Token Authentication:** + +- `authToken` - Authorization token for InfluxDB server. + +### Example + +Below is an example of how to deploy a scaled object with the `InfluxDB` scale trigger. + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: influxdb-scaledobject + namespace: my-project +spec: + scaleTargetRef: + name: nginx-worker + triggers: + - type: influxdb + metadata: + serverURL: http://influxdb:8086 + organizationNameFromEnv: INFLUXDB_ORG_NAME + thresholdValue: '4' + query: | + from(bucket: "bucket-of-interest") + |> range(start: -12h) + |> filter(fn: (r) => r._measurement == "stat") + authTokenFromEnv: INFLUXDB_AUTH_TOKEN +``` diff --git a/content/docs/2.7/scalers/kubernetes-workload.md b/content/docs/2.7/scalers/kubernetes-workload.md new file mode 100644 index 00000000000..1d4e38c5dc7 --- /dev/null +++ b/content/docs/2.7/scalers/kubernetes-workload.md @@ -0,0 +1,48 @@ ++++ +title = "Kubernetes Workload" +layout = "scaler" +availability = "v2.4+" +maintainer = "Community" +description = "Scale applications based on the count of running pods that match the given selectors." +go_file = "kubernetes_workload_scaler" ++++ + +### Trigger Specification + +```yaml +triggers: +- type: kubernetes-workload + metadata: + podSelector: 'app=backend' + value: '1' +``` + +**Parameter list:** + +- `podSelector` - [Label selector](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors) that will be used to get the pod count. It supports multiple selectors split by a comma character (`,`). It also supports [set-based requirements](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#set-based-requirement) and a mix of them. +- `value` - Target relation between the scaled workload and the amount of pods which matches the selector. It will be calculated following this formula: `relation = (pods which match selector) / (scaled workload pods)`. + +> 💡 **Note:** The search scope is limited to the namespace where the `ScaledObject` is deployed. + +The count excludes terminated pods, i.e. [pod status](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#podstatus-v1-core) `phase` equals `Succeeded` or `Failed`. + +### Authentication Parameters + +The own KEDA's identity is used to list the pods, so no extra configuration is needed here. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: workload-scaledobject +spec: + scaleTargetRef: + name: workload-deployment + triggers: + - type: kubernetes-workload + metadata: + podSelector: 'app=backend, deploy notin (critical, monolith)' + value: '3'` +``` diff --git a/content/docs/2.7/scalers/liiklus-topic.md b/content/docs/2.7/scalers/liiklus-topic.md new file mode 100644 index 00000000000..19da5243de1 --- /dev/null +++ b/content/docs/2.7/scalers/liiklus-topic.md @@ -0,0 +1,60 @@ ++++ +title = "Liiklus Topic" +layout = "scaler" +availability = "v1.0+" +maintainer = "Community" +description = "Scale applications based on Liiklus Topic." +go_file = "liiklus_scaler" ++++ + +### Trigger Specification + +This specification describes the `liiklus` trigger for Liiklus Topic. + +```yaml +triggers: +- type: liiklus + metadata: + # Required + address: localhost:6565 + group: my-group + topic: test-topic + # Optional + lagThreshold: "50" + groupVersion: 1 +``` + +**Parameter list:** + +- `address` - Address of the gRPC liiklus API endpoint. +- `group` - Name of consumer group. +- `topic` - Topic to monitor and scale based on `lagThreshold`. +- `lagThreshold` - Value to trigger scaling actions for. (Default: `10`, Optional) +- `groupVersion` - Version of the group to consider when looking at messages. See [docs](https://github.com/bsideup/liiklus/blob/22efb7049ebcdd0dcf6f7f5735cdb5af1ae014de/app/src/test/java/com/github/bsideup/liiklus/GroupVersionTest.java). (Default: `0`, Optional) + +### Authentication Parameters + +Not supported yet. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: liiklus-scaledobject + namespace: default +spec: + scaleTargetRef: + name: function-deployment + pollingInterval: 30 + triggers: + - type: liiklus + metadata: + # Required + address: localhost:6565 + group: my-group # Make sure that this consumer group name is the same one as the one that is consuming topics + topic: test-topic + # Optional + lagThreshold: "50" # Default value is set to 10 +``` diff --git a/content/docs/2.7/scalers/memory.md b/content/docs/2.7/scalers/memory.md new file mode 100644 index 00000000000..47cbd42c251 --- /dev/null +++ b/content/docs/2.7/scalers/memory.md @@ -0,0 +1,50 @@ ++++ +title = "Memory" +layout = "scaler" +availability = "v2.0+" +maintainer = "Community" +description = "Scale applications based on memory metrics." +go_file = "cpu_memory_scaler" ++++ + +> **Notice:** +> - This scaler will never scale to 0 and even when user defines multiple scaler types (eg. Kafka + cpu/memory, or Prometheus + cpu/memory), the deployment will never scale to 0. +> - This scaler only applies to ScaledObject, not to Scaling Jobs. + +### Trigger Specification + +This specification describes the `memory` trigger that scales based on memory metrics. + +```yaml +triggers: +- type: memory + metadata: + # Required + type: Utilization/ AverageValue + value: "60" +``` + +**Parameter list:** + +- `type` - Type of metric to use. Options are `Utilization`, or `AverageValue`. +- `value` - Value to trigger scaling actions for: + - When using `Utilization`, the target value is the average of the resource metric across all relevant pods, represented as a percentage of the requested value of the resource for the pods. + - When using `AverageValue`, the target value is the target value of the average of the metric across all relevant pods (quantity). + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: memory-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + triggers: + - type: memory + metadata: + type: Utilization + value: "50" +``` diff --git a/content/docs/2.7/scalers/metrics-api.md b/content/docs/2.7/scalers/metrics-api.md new file mode 100644 index 00000000000..0a57f74781b --- /dev/null +++ b/content/docs/2.7/scalers/metrics-api.md @@ -0,0 +1,295 @@ ++++ +title = "Metrics API" +layout = "scaler" +availability = "v2.0+" +maintainer = "Community" +description = "Scale applications based on a metric provided by an API" +go_file = "metrics_api_scaler" ++++ + +### Trigger Specification + +This specification describes the `metrics-api` trigger that scales based on a metric value provided by an API. + +This scaler allows users to utilize **any existing APIs** as a metric provider. + +Here is an example of trigger configuration using metrics-api scaler: + +```yaml +triggers: +- type: metrics-api + metadata: + targetValue: "8" + url: "http://api:3232/api/v1/stats" + valueLocation: "components.worker.tasks" +``` + +**Parameter list:** + +- `url` - Full URL of the API operation to call to get the metric value (eg. `http://app:1317/api/v1/stats`). +- `valueLocation` - [GJSON path notation](https://github.com/tidwall/gjson#path-syntax) to refer to the field in the payload containing the metric value. +- `targetValue` - Target value to scale on. When the metric provided by the API is equal or higher to this value, KEDA will start scaling out. When the metric is 0 or less, KEDA will scale down to 0. + +### Authentication Parameters + +Metrics Scaler API supported three types of authentication - API Key based authentication, basic authentication and TLS +authentication. + +You can use `TriggerAuthentication` CRD to configure the authentication. Specify `authMode` and other trigger parameters + along with secret credentials in `TriggerAuthentication` as mentioned below: + +**API Key based authentication:** +- `authMode`: It must be set to `apiKey` in case of API key Authentication. Specify this in trigger configuration. +- `method` - This specifies the possible methods API Key based authentication supports. Possible values are `header` and `query`. `header` is the default method. Specify this in trigger configuration. +- `keyParamName` - This is either header key or query param used for passing apikey. Default header is `X-API-KEY` and default query param is `api_key`. Specify this in trigger configuration. If your implementation has different key, please specify it here. +- `apiKey` - API Key needed for authentication. + +**Basic authentication:** +- `authMode` - It must be set to `basic` in case of Basic Authentication. Specify this in trigger configuration. +- `username` - Provide the username to be used for basic authentication. +- `password` - Provide the password to be used for authentication. (Optional) + +**TLS authentication:** +- `authMode` - It must be set to `tls` in case of TLS Authentication. Specify this in trigger configuration. +- `ca` - Certificate authority file for TLS client authentication. +- `cert` - Certificate for client authentication. +- `key` - Key for client authentication. (Optional) + +**Bearer authentication:** +- `authMode` - It must be set to `bearer` in case of Bearer Authentication. Specify this in trigger configuration. +- `token` - Token that should be placed in the `Authorization` header. The header will be `Authorization: Bearer {token}`. + +### Example + +Here is a full example of scaled object definition using Metric API trigger: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: http-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: metrics-api + metadata: + targetValue: "7" + url: "http://api:3232/components/stats" + valueLocation: 'components.worker.tasks' +``` + +When checking current metric Metrics API scaler sends GET request to provided `url` and then uses `valueLocation` +to access the value in response's payload. + +The above example expects that the API endpoint will return response similar to this one: +```json +{ + "components": { + "worker": { + "tasks": 12, + ... + }, + ... + }, + ... +} +``` +Assuming such response, Metrics API trigger will figure out that current metric value is 12. + +> 💡 **NOTE:**The value of the metric can either be an integral unquoted json number type (e.g. 123). Or a quantity (e.g. "123", "1.23", "10Mi"). + +Here is an example of a metric scaler with API Key based authentication, + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-metric-api-secret + namespace: default +data: + apiKey: "APIKEY" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-metric-api-creds + namespace: default +spec: + secretTargetRef: + - parameter: apiKey + name: keda-metric-api-secret + key: apiKey +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: http-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: metrics-api + metadata: + targetValue: "7" + url: "http://api:3232/components/stats" + valueLocation: 'components.worker.tasks' + authMode: "apiKey" + method: "query" + keyParamName: "QUERY_KEY" + authenticationRef: + name: keda-metric-api-creds +``` + +Here is an example of a metric scaler with Bearer Authentication, define the `Secret` and `TriggerAuthentication` as follows + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-metric-api-secret + namespace: default +data: + token: "PlaceYourTokenHere" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-metric-api-creds + namespace: default +spec: + secretTargetRef: + - parameter: token + name: keda-metric-api-secret + key: token +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: http-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: metrics-api + metadata: + targetValue: "7" + url: "http://api:3232/components/stats" + valueLocation: 'components.worker.tasks' + authMode: "bearer" + authenticationRef: + name: keda-metric-api-creds +``` + +Here is an example of a metric scaler with Basic Authentication, define the `Secret` and `TriggerAuthentication` as follows + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-metric-api-secret + namespace: default +data: + username: "username" + password: "password" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-metric-api-creds + namespace: default +spec: + secretTargetRef: + - parameter: username + name: keda-metric-api-secret + key: username + - parameter: password + name: keda-metric-api-secret + key: password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: http-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: metrics-api + metadata: + targetValue: "7" + url: "http://api:3232/components/stats" + valueLocation: 'components.worker.tasks' + authMode: "basic" + authenticationRef: + name: keda-metric-api-creds +``` + + +Here is an example of a metric scaler with TLS Authentication, define the `Secret` and `TriggerAuthentication` as follows + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-metric-api-secret + namespace: default +data: + cert: "cert" + key: "key" + ca: "ca" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-metric-api-creds + namespace: default +spec: + secretTargetRef: + - parameter: cert + name: keda-metric-api-secret + key: cert + - parameter: key + name: keda-metric-api-secret + key: key + - parameter: ca + name: keda-metric-api-secret + key: ca +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: http-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: metrics-api + metadata: + targetValue: "7" + url: "http://api:3232/components/stats" + valueLocation: 'components.worker.tasks' + authMode: "tls" + authenticationRef: + name: keda-metric-api-creds +``` diff --git a/content/docs/2.7/scalers/mongodb.md b/content/docs/2.7/scalers/mongodb.md new file mode 100644 index 00000000000..53ed2d941e8 --- /dev/null +++ b/content/docs/2.7/scalers/mongodb.md @@ -0,0 +1,159 @@ ++++ +title = "MongoDB" +layout = "scaler" +maintainer = "Community" +description = "Scale applications based on MongoDB queries." +availability = "v2.1+" +go_file = "mongo_scaler" ++++ + +### Trigger Specification + +This specification describes the `mongodb` trigger that scales based on result of MongoDB query. + +```yaml +triggers: + - type: mongodb + metadata: + # name of an environment variable containing a valid MongoDB connection string + connectionStringFromEnv: MongoDB_CONNECTION_STRING + # Required: database name + dbName: "test" + # Required: collection name + collection: "test_collection" + # Required: query expr, used by filter data + query: '{"region":"eu-1","state":"running","plan":"planA"}' + # Required: according to the number of query result, to scale the TargetRef + queryValue: "1" + # Optional: The generated metric name would be mongodb-global-metric. Here mongodb- use as a prefix for metric name + metricName: "global-metric" +``` + +Alternatively, you can configure connection parameters explicitly instead of providing a connection string: + +```yaml +triggers: + - type: mongodb + metadata: + # host name of the MongoDB server. Example of mongodb service: "mongodb-svc..svc.cluster.local" + host: mongodb-svc.default.svc.cluster.local + # port number of the MongoDB server. + port: "27017" + # username credential for connecting to the MongoDB server + username: test_user + # name of an environment variable containing a valid password for connecting to the MongoDB server + passwordFromEnv: MongoDB_Password + # Required: database name + dbName: "test" + # Required: collection name + collection: "test_collection" + # Required: query expr, used by filter data + query: '{"region":"eu-1","state":"running","plan":"planA"}' + # Required: according to the number of query result, to scale the TargetRef + queryValue: "1" + # Optional: The generated metric name would be mongodb-global-metric. Here mongodb- use as a prefix for metric name. + metricName: "global-metric" +``` + +>**NOTE:** If **metricName** is not set, then one is generated based on trigger index and the `collection`, for example: **s0-mongodb-test_collection** + +**Parameter list:** + +The `mongodb` trigger always requires the following information: + +- `dbName` - Name of the database. +- `collection` - Name of the collection. +- `query` - A MongoDB query that should return single numeric value. +- `queryValue` - A threshold that will define when scaling should occur. + +To connect to the MongoDB server, you can provide either: + +- `connectionStringFromEnv` - The name of an environment variable containing a valid MongoDB connection string for connecting to the MongoDB server. + +Or provide more detailed connection parameters explicitly (a connection string will be generated for you at runtime): + +- `host` - The host name of the MongoDB server. +- `port` - The port number of the MongoDB server. +- `username` - Username to authenticate with to MongoDB database. +- `passwordFromEnv` - The name of an environment variable containing the password credential for connecting to the MongoDB server. + +When configuring with a connection string, you can use this URL format: + +``` +mongodb://:@mongodb-svc..svc.cluster.local:27017/ +``` + +You can also optionally assign a name to the metric using the `metricName` value. If not specified, the `metricName` will be generated automatically based on trigger index and collection name. For example: **s1-mongodb-test_collection**. The value will be prefixed with `s{triggerIndex}-mongodb-` . + +### Authentication Parameters + +As an alternative to environment variables, You can authenticate with the MongoDB server by using connection string or password authentication via `TriggerAuthentication` or `ClusterTriggerAuthentication` configuration. + +**Connection String Authentication:** + +- `connectionString` - Connection string for MongoDB server. + +**Password Authentication:** + +- `host` - The host name of the MongoDB server. +- `port` - The port number of the MongoDB server. +- `username` - Username to authenticate with to MongoDB database. +- `password` - Password for the configured user to login to MongoDB server. +- `dbName` - Name of the database. + +### Example + +Here is an example of how to deploy a scaled Job with the `MongoDB` scale trigger which uses `TriggerAuthentication`. + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledJob +metadata: + name: mongodb-job +spec: + jobTargetRef: + template: + spec: + containers: + - name: mongodb-update + image: 1314520999/mongodb-update:latest + args: + - --connectStr=mongodb://test_user:test_password@mongoDB-svc.mongoDB.svc.cluster.local:27017/test + - --dataBase=test + - --collection=test_collection + imagePullPolicy: IfNotPresent + restartPolicy: Never + backoffLimit: 1 + pollingInterval: 30 # Optional. Default: 30 seconds + maxReplicaCount: 30 # Optional. Default: 100 + successfulJobsHistoryLimit: 0 # Optional. Default: 100. How many completed jobs should be kept. + failedJobsHistoryLimit: 10 # Optional. Default: 100. How many failed jobs should be kept. + triggers: + - type: mongodb + metadata: + dbName: "test" + collection: "test_collection" + query: '{"region":"eu-1","state":"running","plan":"planA"}' + queryValue: "1" + metricName: "global-metric" + authenticationRef: + name: mongodb-trigger +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: mongodb-trigger +spec: + secretTargetRef: + - parameter: connectionString + name: mongodb-secret + key: connect +--- +apiVersion: v1 +kind: Secret +metadata: + name: mongodb-secret +type: Opaque +data: + connect: bW9uZ29kYjovL3Rlc3RfdXNlcjp0ZXN0X3Bhc3N3b3JkQG1vbmdvZGItc3ZjLm1vbmdvREIuc3ZjLmNsdXN0ZXIubG9jYWw6MjcwMTcvdGVzdA== +``` diff --git a/content/docs/2.7/scalers/mssql.md b/content/docs/2.7/scalers/mssql.md new file mode 100644 index 00000000000..feeafca21ad --- /dev/null +++ b/content/docs/2.7/scalers/mssql.md @@ -0,0 +1,126 @@ ++++ +title = "MSSQL" +layout = "scaler" +availability = "v2.2+" +maintainer = "Microsoft" +description = "Scale applications based on Microsoft SQL Server (MSSQL) query results." +go_file = "mssql_scaler" ++++ + +### Trigger Specification + +This specification describes the `mssql` trigger that scales based on the results of a [Microsoft SQL Server](https://www.microsoft.com/sql-server/) (MSSQL) query result. This trigger supports local [MSSQL containers](https://hub.docker.com/_/microsoft-mssql-server) as well as SQL Server endpoints hosted in the cloud, such as [Azure SQL Database](https://azure.microsoft.com/services/sql-database/). + +```yaml +triggers: +- type: mssql + metadata: + connectionStringFromEnv: MSSQL_CONNECTION_STRING + query: "SELECT COUNT(*) FROM backlog WHERE state='running' OR state='queued'" + targetValue: 1 + metricName: backlog_process_count # optional - the generated value would be `mssql-{sha256hash}` +``` + +> 💡 **NOTE:** The connection string format supported by this scaler has some incompatibilities with connection string formats supported by other platforms, like .NET. For example, the MSSQL instance's port number must be separated into its own `Port` property instead of adding it to the `Server` property. You can learn more about all the supported connection string formats for this mssql scaler [here](https://github.com/denisenkom/go-mssqldb#the-connection-string-can-be-specified-in-one-of-three-formats). + +Alternatively, you configure connection parameters explicitly instead of providing a connection string: + +```yaml +triggers: +- type: mssql + metadata: + username: "kedaUser" + passwordFromEnv: MSSQL_PASSWORD + host: mssqlinst.namespace.svc.cluster.local + port: "1433" # optional + database: test_db_name + query: "SELECT COUNT(*) FROM backlog WHERE state='running' OR state='queued'" + targetValue: 1 + metricName: backlog_process_count # optional - the generated value would be `mssql-test_db_name` +``` + +The `mssql` trigger always requires the following information: + +- `query` - A [T-SQL](https://docs.microsoft.com/sql/t-sql/language-reference) query that returns a single numeric value. This can be a regular query or the name of a stored procedure. +- `targetValue` - A threshold that is used as `targetAverageValue` in the Horizontal Pod Autoscaler (HPA). + +To connect to the MSSQL instance, you can provide either: + +- `connectionStringFromEnv` - The name of an environment variable containing a valid MSSQL connection string. + +Or provide more detailed connection parameters explicitly (a connection string will be generated for you at runtime): + +- `host` - The hostname of the MSSQL instance endpoint. +- `port` - The port number of the MSSQL instance endpoint. (Default: 1433, Optional) +- `database` - The name of the database to query. +- `username` - The username credential for connecting to the MSSQL instance. +- `passwordFromEnv` - The name of an environment variable containing the password credential for connecting to the MSSQL instance. + +When configuring with a connection string, you can use either a URL format (note the URL encoding of special characters): + +``` +sqlserver://user1:Password%231@example.database.windows.net:1433?database=AdventureWorks +``` + +Or the more traditional ADO format: + +``` +Server=example.database.windows.net;Port=1433;Database=AdventureWorks;Persist Security Info=False;User ID=user1;Password=Password#1;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30; +``` + +You can also optionally assign a name to the metric using the `metricName` value. If not specified, the `metricName` will be generated automatically based on trigger index and the `database` value (if specified), or the `host` value, or will be in the form `s{triggerIndex}-mssql-{sha256hash}` where `{sha256hash}` is a SHA-256 hash of the connection string. + +### Authentication parameters + +As an alternative to using environment variables, you can authenticate with the MSSQL instance using connection string or password authentication via `TriggerAuthentication` configuration. + +**Connection string authentication:** + +- `connectionString` - The connection string for the MSSQL instance. + +**Password authentication:** + +- `host` - The hostname of the MSSQL instance endpoint. +- `port` - The port number of the MSSQL instance endpoint. (default 1433) +- `database` - The name of the database to query. +- `username` - The username credential for connecting to the MSSQL instance. +- `password` - The password credential for connecting to the MSSQL instance. + +### Example + +The following is an example of how to deploy a scaled object with the `mssql` scale trigger that uses `TriggerAuthentication` and a connection string. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: mssql-secrets +type: Opaque +data: + mssql-connection-string: U2VydmVyPWV4YW1wbGUuZGF0YWJhc2Uud2luZG93cy5uZXQ7cG9ydD0xNDMzO0RhdGFiYXNlPUFkdmVudHVyZVdvcmtzO1BlcnNpc3QgU2VjdXJpdHkgSW5mbz1GYWxzZTtVc2VyIElEPXVzZXIxO1Bhc3N3b3JkPVBhc3N3b3JkIzE7RW5jcnlwdD1UcnVlO1RydXN0U2VydmVyQ2VydGlmaWNhdGU9RmFsc2U7 # base64 encoded value of MSSQL connectionString of format "Server=example.database.windows.net;port=1433;Database=AdventureWorks;Persist Security Info=False;User ID=user1;Password=Password#1;Encrypt=True;TrustServerCertificate=False;" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-mssql-secret +spec: + secretTargetRef: + - parameter: connectionString + name: mssql-secrets + key: mssql-connection-string +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: mssql-scaledobject +spec: + scaleTargetRef: + name: consumer # e.g. the name of the resource to scale + triggers: + - type: mssql + metadata: + targetValue: 1 + query: "SELECT COUNT(*) FROM backlog WHERE state='running' OR state='queued'" + authenticationRef: + name: keda-trigger-auth-mssql-secret +``` diff --git a/content/docs/2.7/scalers/mysql.md b/content/docs/2.7/scalers/mysql.md new file mode 100644 index 00000000000..dbef89f3899 --- /dev/null +++ b/content/docs/2.7/scalers/mysql.md @@ -0,0 +1,93 @@ ++++ +title = "MySQL" +layout = "scaler" +availability = "v1.2+" +maintainer = "Community" +description = "Scale applications based on MySQL query result." +go_file = "mysql_scaler" ++++ + +### Trigger Specification + +This specification describes the `mysql` trigger that scales based on result of MySQL query. + +The trigger always requires the following information: + +- `query` - A MySQL query that should return single numeric value. +- `queryValue` - A threshold that is used as `targetAverageValue` in HPA. + +To provide information about how to connect to MySQL you can provide: + +- `connectionStringFromEnv` - MySQL connection string that should point to environment variable with valid value. + +Or provide more detailed information: + +- `host` - The host of the MySQL server. +- `port` - The port of the MySQL server. +- `dbName` - Name of the database. +- `username` - Username to authenticate with to MySQL database. +- `passwordFromEnv` - Password for the given user, this should be blank (no password) or point to an environment variable with the password. + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `hostFromEnv` - The host of the MySQL server, similar to `host`, but reads it from an environment variable on the scale target. +- `portFromEnv` - The port of the MySQL server, similar to `port`, but reads it from an environment variable on the scale target. + +### Authentication Parameters + +You can authenticate by using connection string or password authentication. + +**Connection String Authentication:** + +- `connectionString` - Connection string for MySQL database. + +**Password Authentication:** + +- `host` - The host of the MySQL server. +- `port` - The port of the MySQL server. +- `dbName` - Name of the database. +- `username` - Username to authenticate with to MySQL database. +- `password` - Password for configured user to login to MySQL database. +variables. + +### Example + +Here is an example of how to deploy a scaled object with the `mysql` scale trigger which uses `TriggerAuthentication`. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: mysql-secrets + namespace: my-project +type: Opaque +data: + mysql_conn_str: dXNlckB0Y3AobXlzcWw6MzMwNikvc3RhdHNfZGI= # base64 encoded value of mysql connectionString of format user@tcp(mysql:3306)/stats_db +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-mysql-secret + namespace: my-project +spec: + secretTargetRef: + - parameter: connectionString + name: mysql-secrets + key: mysql_conn_str +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: mysql-scaledobject + namespace: my-project +spec: + scaleTargetRef: + name: worker + triggers: + - type: mysql + metadata: + queryValue: "4" + query: "SELECT CEIL(COUNT(*) / 6) FROM task_instance WHERE state='running' OR state='queued'" + authenticationRef: + name: keda-trigger-auth-mysql-secret +``` diff --git a/content/docs/2.7/scalers/nats-streaming.md b/content/docs/2.7/scalers/nats-streaming.md new file mode 100644 index 00000000000..a617d1d4714 --- /dev/null +++ b/content/docs/2.7/scalers/nats-streaming.md @@ -0,0 +1,107 @@ ++++ +title = "NATS Streaming" +layout = "scaler" +availability = "v1.0+" +maintainer = "Community" +description = "Scale applications based on NATS Streaming." +go_file = "stan_scaler" ++++ + +### Trigger Specification + +This specification describes the `stan` trigger for NATS Streaming. + +```yaml +triggers: +- type: stan + metadata: + natsServerMonitoringEndpoint: "stan-nats-ss.stan.svc.cluster.local:8222" + queueGroup: "grp1" + durableName: "ImDurable" + subject: "Test" + lagThreshold: "10" +``` + +**Parameter list:** + +- `natsServerMonitoringEndpoint` - Location of the Nats Streaming monitoring endpoint. +- `queueGroup` - Name of queue group of the subscribers. +- `durableName` - Name of durability used by subscribers. +- `subject` - Name of the channel. +- `lagThreshold` - Average target value to trigger scaling actions. + +### Authentication Parameters + +You can authenticate with the NATS streaming server by using connection string authentication via `TriggerAuthentication` configuration. + +- `natsServerMonitoringEndpoint` - Location of the NATS Streaming monitoring endpoint. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: stan-scaledobject + namespace: example +spec: + pollingInterval: 10 # Optional. Default: 30 seconds + cooldownPeriod: 30 # Optional. Default: 300 seconds + minReplicaCount: 0 # Optional. Default: 0 + maxReplicaCount: 30 # Optional. Default: 100 + scaleTargetRef: + name: gonuts-sub + triggers: + - type: stan + metadata: + natsServerMonitoringEndpoint: "stan-nats-ss.stan.svc.cluster.local:8222" + queueGroup: "grp1" + durableName: "ImDurable" + subject: "Test" + lagThreshold: "10" +``` +#### Example with TriggerAuthentication: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: stan-secret + namespace: example +type: Opaque +data: + stan_endpoint: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-stan-secret + namespace: example +spec: + secretTargetRef: + - parameter: natsServerMonitoringEndpoint + name: stan-secret + key: stan_endpoint +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: stan-scaledobject + namespace: example +spec: + pollingInterval: 10 # Optional. Default: 30 seconds + cooldownPeriod: 30 # Optional. Default: 300 seconds + minReplicaCount: 0 # Optional. Default: 0 + maxReplicaCount: 30 # Optional. Default: 100 + scaleTargetRef: + name: gonuts-sub + triggers: + - type: stan + metadata: + queueGroup: "grp1" + durableName: "ImDurable" + subject: "Test" + lagThreshold: "10" + authenticationRef: + name: keda-trigger-auth-stan-secret +``` \ No newline at end of file diff --git a/content/docs/2.7/scalers/new-relic.md b/content/docs/2.7/scalers/new-relic.md new file mode 100644 index 00000000000..a5153ca2358 --- /dev/null +++ b/content/docs/2.7/scalers/new-relic.md @@ -0,0 +1,92 @@ ++++ +title = "New Relic" +availability = "2.6+" +maintainer = "Community" +description = "Scale applications based on New Relic NRQL" +layout = "scaler" +go_file = "newrelic_scaler" ++++ + +### Trigger Specification + +This specification describes the `new-relic` trigger that scales based on a New Relic metric. + +```yaml +triggers: + - type: new-relic + metadata: + # Required: Account - Subaccount to run the query on + account: 1234567 + # Required: QueryKey - Api key to connect to New Relic + queryKey: "NRAK-xxxxxxxxxxxxxxxxxxxxxxxxxxx" + # Optional: nrRegion - Region to query data for. Default value is US. + region: "US" + # Optional: noDataError - If the query returns no data should this be treated as an error. Default value is false. + noDataError: "true" + # Required: nrql + nrql: "SELECT average(duration) from Transaction where appName='SITE' TIMESERIES" + # Required: threshold + threshold: 100 +``` + +**Parameter list:** + +- `account` - The account within New Relic that the request should be targeted against. +- `queryKey` - The API key that will be leveraged to connect to New Relic and make requests. [official documentation](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/) +- `region` - The region to connect to for the New Relic apis. (Values: `LOCAL`, `EU`, `STAGING`, `US`, Default: `US`, Optional) +- `noDataError` - Should queries that return nodata be treated as an error (Values: `true`, `false`, Default: `false`, Optional) +- `nrql` - The New Relic query that will be run to get the data requested. [official documentation](https://docs.newrelic.com/docs/query-your-data/nrql-new-relic-query-language/get-started/introduction-nrql-new-relics-query-language/) +- `threshold` - A threshold that is used as the `targetAverageValue` in the HPA configuration. + +### Authentication Parameters + +You can use `TriggerAuthentication` CRD to configure the authentication with a `queryKey`. + +**Authentication:** + +- `queryKey` - The API key that will be leveraged to connect to New Relic and make requests. [official documentation](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/) + +### Example + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-relic-secret + namespace: my-project +type: Opaque +data: + apiKey: TlJBSy0xMjM0NTY3ODkwMTIzNDU2Nwo= # base64 encoding of the new relic api key NRAK-12345678901234567 +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-new-relic + namespace: my-project +spec: + secretTargetRef: + - parameter: queryKey + name: new-relic-secret + key: apiKey + +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: newrelic-scaledobject + namespace: keda +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: new-relic + metadata: + account: 1234567 + region: "US" + nrql: "SELECT average(duration) from Transaction where appName='SITE' TIMESERIES" + noDataError: "true" + threshold: 1000 + authenticationRef: + name: keda-trigger-auth-new-relic +``` diff --git a/content/docs/2.7/scalers/openstack-metric.md b/content/docs/2.7/scalers/openstack-metric.md new file mode 100644 index 00000000000..81781fc79a4 --- /dev/null +++ b/content/docs/2.7/scalers/openstack-metric.md @@ -0,0 +1,172 @@ ++++ +title = "OpenStack Metric" +layout = "scaler" +availability = "v2.3+" +maintainer = "Community" +description = "Scale applications based on a threshold reached by a specific measure from OpenStack Metric API." +go_file = "openstack_metrics_scaler" ++++ + +### Trigger Specification + +This specification describes the `openstack-metric` trigger for OpenStack metrics. +> The OpenStack metric API follows [Gnocchi](https://gnocchi.xyz/) API. Attempt to the fact that Gnocchi API is an open-source time series database embedded in OpenStack system and every parameter on OpenStack Metric API follows its patterns but you don't need to reference anything from Gnocchi. It scales based on a specific measure from a given resource metric. +It's highly recommended to check [Gnocchi](https://gnocchi.xyz/) docs. + +```yaml +triggers: +- type: openstack-metric + metadata: + metricsURL: http://localhost:8041/v1/metric #required + metricID: 003bb589-166d-439d-8c31-cbf098d863de #required + aggregationMethod: "mean" #required + granularity: 300 #required (seconds) + threshold: 1250 #required + timeout: 30 #optional +``` + +> Protocol (http or https) should always be provided when specifying URLs + +**Parameter list:** +- `metricsURL` - The URL to check for the metrics API, based. It must contain the hostname, the metric port, the API version, and the resource ID. The pattern is: `http://:///metric`. +- `metricID` - The Id of the intendend metric. +- `aggregationMethod` - The aggregation method that will be used to calculate metrics, it must follows the configured possible metrics derived from gnocchi API like: `mean`, `min`, `max`, `std`, `sum`, `count`, the complete aggregation methods list can be found [here](https://gnocchi.xyz/rest.html#archive-policy). +- `granularity` - The configured granularity from metric collection in seconds. it must follow the same value configured in OpenStack, but it must be coutned in seconds. Sample: If you have a 5 minutes time window granularity defined, so you must input a value of 300 seconds (5*60). +- `threshold` - The target value that, when reached, will scale the application. +- `timeout` - The timeout, in seconds, for the HTTP client requests that will query the Metric API. (Default: `30`, Optional) + +### Authentication Parameters + +To authenticate, this scaler uses tokens. Tokens are automatically retrieved by the scaler from [Keystone](https://docs.openstack.org/keystone/latest/), the official OpenStack Identity Provider. You can provide your credentials using Secrets either by using the "password" method or the "application credentials" method. Both cases use `TriggerAuthentication`. + +#### Password + +- `authURL` - The Keystone authentication URL. The pattern is: `http://://`. +- `userID` - The OpenStack project user ID. +- `password` - The password for the provided user. +- `projectID` - The OpenStack project ID. + +#### Application Credentials + +- `authURL` - The Keystone authentication URL. The pattern is: `http://://`. +- `appCredentialID` - The Application Credential ID. +- `appCredentialSecret` - The Application Credential secret. + +### Example + +#### Password method + +Here is an example of how to deploy a scaled object with the `openstack-metric` scale trigger which uses `TriggerAuthentication` and the Password method from above. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: openstack-secret-password + namespace: default +type: Opaque +data: + authURL: aHR0cDovL2xvY2FsaG9zdDo1MDAwL3YzLw== + userID: MWYwYzI3ODFiNDExNGQxM2E0NGI4ODk4Zjg1MzQwYmU= + password: YWRtaW4= + projectID: YjE2MWRjNTE4Y2QyNGJkYTg0ZDk0ZDlhMGU3M2ZjODc= +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: metrics-password-trigger-authentication + namespace: default +spec: + secretTargetRef: + - parameter: authURL + name: openstack-secret-password + key: authURL + - parameter: userID + name: openstack-secret-password + key: userID + - parameter: password + name: openstack-secret-password + key: password + - parameter: projectID + name: openstack-secret-password + key: projectID +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: openstack-metric-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + pollingInterval: 10 + cooldownPeriod: 10 + minReplicaCount: 0 + triggers: + - type: openstack-metric + metadata: + metricsURL: http://localhost:8041/v1/metric + metricID: faf01aa5-da88-4929-905d-b83fbab46771 + aggregationMethod: "mean" + granularity: 300 + threshold: 1250 + timeout: 30 + authenticationRef: + name: openstack-metric-password-trigger-authentication +``` + +#### Application Credentials method + +You can also use the Application Credentials method. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: openstack-secret-appcredentials + namespace: default +type: Opaque +data: + authURL: aHR0cDovL2xvY2FsaG9zdDo1MDAwL3YzLw== + appCredentialID: OWYyY2UyYWRlYmFkNGQxNzg0NTgwZjE5ZTljMTExZTQ= + appCredentialSecret: LVdSbFJBZW9sMm91Z3VmZzNEVlBqcll6aU9za1pkZ3c4Y180XzRFU1pZREloT0RmajJkOHg0dU5yb3NudVIzWmxDVTZNLTVDT3R5NDFJX3M5R1N5Wnc= +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: openstack-metric-appcredentials-trigger-authentication + namespace: default +spec: + secretTargetRef: + - parameter: authURL + name: openstack-secret-appcredentials + key: authURL + - parameter: appCredentialID + name: openstack-secret-appcredentials + key: appCredentialID + - parameter: appCredentialSecret + name: openstack-secret-appcredentials + key: appCredentialSecret +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: openstack-metric-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + pollingInterval: 10 + cooldownPeriod: 10 + minReplicaCount: 0 + triggers: + - type: openstack-metric + metadata: + metricsURL: http://localhost:8041/v1/metric + metricID: faf01aa5-da88-4929-905d-b83fbab46771 + aggregationMethod: "mean" + granularity: 300 + threshold: 1250 + authenticationRef: + name: openstack-metric-appcredentials-trigger-authentication +``` diff --git a/content/docs/2.7/scalers/openstack-swift.md b/content/docs/2.7/scalers/openstack-swift.md new file mode 100644 index 00000000000..b2a276f2214 --- /dev/null +++ b/content/docs/2.7/scalers/openstack-swift.md @@ -0,0 +1,178 @@ ++++ +title = "OpenStack Swift" +layout = "scaler" +availability = "v2.1+" +maintainer = "Community" +description = "Scale applications based on the count of objects in a given OpenStack Swift container." +go_file = "openstack_swift_scaler" ++++ + +### Trigger Specification + +This specification describes the `openstack-swift` trigger for OpenStack Swift containers. It scales based on the count of objects in a given container. + +```yaml +triggers: +- type: openstack-swift + metadata: + containerName: my-container # Required + swiftURL: http://localhost:8080/v1/b161dc815cd24bda84d94d9a0e73cf78 # Optional + objectCount: "2" # Optional + objectPrefix: "my-prefix" # Optional + objectDelimiter: "/" # Optional + objectLimit: "10" # Optional + onlyFiles: "true" # Optional + timeout: "2" # Optional +``` + +> Please, always provide the protocol (http or https) when specifying URLs. + +**Parameter list:** + +- `swiftURL` - The URL to query the Swift API. If not provided, the scaler will try to find the Swift public URL for a certain region, using the OpenStack catalog, which is returned when requesting an authentication token. (Optional) +- `containerName` - Name of Swift container in an OpenStack account. +- `objectCount` - Average target value to trigger scaling actions. (Default: `2`, Optional) +- `objectPrefix` - Prefix for the object. Only objects with this prefix will be returned. Use this prefix to specify sub-paths for the objects. (Default: `""`, Optional) +- `objectDelimiter` - Delimiter for identifying the object prefix. It is the character used to split object names. (Default: `""`, Optional) +- `objectLimit` - The maximum number of objects returned by the API. By default, the Swift API only returns up to 10000 names. (Default: `""`, Optional) +- `onlyFiles` - Specifies if the scaler should be triggered only by the number of files, without considering folders. Inside a container, one can have files and folders. Folders (empty or not) are counted as objects, just as files are. If one wants to scale based on only files, this parameter must be set to `true`. (Values: `true`, `false`, Default: `false`, Optional) +- `timeout` - The timeout, in seconds, for the HTTP client requests that will query the Swift API. (Default: `30`, Optional) + +For more information about `prefix`, `delimiter`, and `limit`, please, refer to the [Object Store API](https://docs.openstack.org/api-ref/object-store/). + +### Authentication Parameters + +To authenticate, this scaler uses tokens. Tokens are automatically retrieved by the scaler from [Keystone](https://docs.openstack.org/keystone/latest/), the official OpenStack Identity Provider. You can provide your credentials using Secrets either by using the "password" method or the "application credentials" method. Both cases use `TriggerAuthentication`. + +#### Password + +- `authURL` - The Keystone authentication URL. The scaler supports only Keystone v3. You shouldn't include the `/v3` parameter in your URL path. This is done automatically by the scaler. +- `userID` - The OpenStack project user ID. +- `password` - The password for the provided user. +- `projectID` - The OpenStack project ID. +- `regionName` - The OpenStack region name where the Swift service is available. This parameter is not required and is used only when the `swiftURL` is not provided to the scaler. If the region name is not provided, it will look for the first Swift public URL available in the OpenStack catalog. + +#### Application Credentials + +- `authURL` - The Keystone authentication URL. The scaler supports only Keystone v3. You shouldn't include the `/v3` parameter in your URL path. This is done automatically by the scaler. +- `appCredentialID` - The Application Credential ID. +- `appCredentialSecret` - The Application Credential secret. +- `regionName` - The OpenStack region name where the Swift service is available. This parameter is not required and is used only when the `swiftURL` is not provided to the scaler. If the region name is not provided, it will look for the first Swift public URL available in the OpenStack catalog. + +### Example + +#### Password method + +Here is an example of how to deploy a scaled object with the `openstack-swift` scale trigger which uses `TriggerAuthentication` and the Password method from above. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: openstack-secret-password + namespace: default +type: Opaque +data: + authURL: aHR0cDovL2xvY2FsaG9zdDo1MDAwLw== + userID: MWYwYzI3ODFiNDExNGQxM2E0NGI4ODk4Zjg1MzQwYmU= + password: YWRtaW4= + projectID: YjE2MWRjNTE4Y2QyNGJkYTg0ZDk0ZDlhMGU3M2ZjODc= + regionName: Y2FsaWZvcm5pYS0x +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: swift-password-trigger-authentication + namespace: default +spec: + secretTargetRef: + - parameter: authURL + name: openstack-secret-password + key: authURL + - parameter: userID + name: openstack-secret-password + key: userID + - parameter: password + name: openstack-secret-password + key: password + - parameter: projectID + name: openstack-secret-password + key: projectID + - parameter: regionName + name: openstack-secret-password + key: regionName +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: swift-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + pollingInterval: 10 + cooldownPeriod: 10 + minReplicaCount: 0 + triggers: + - type: openstack-swift + metadata: + containerName: my-container + objectCount: "1" + onlyFiles: "true" + authenticationRef: + name: swift-password-trigger-authentication +``` + +#### Application Credentials method + +You can also use the Application Credentials method. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: openstack-secret-appcredentials + namespace: default +type: Opaque +data: + authURL: aHR0cDovL2xvY2FsaG9zdDo1MDAwLw== + appCredentialID: OWYyY2UyYWRlYmFkNGQxNzg0NTgwZjE5ZTljMTExZTQ= + appCredentialSecret: LVdSbFJBZW9sMm91Z3VmZzNEVlBqcll6aU9za1pkZ3c4Y180XzRFU1pZREloT0RmajJkOHg0dU5yb3NudVIzWmxDVTZNLTVDT3R5NDFJX3M5R1N5Wnc= +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: swift-appcredentials-trigger-authentication + namespace: default +spec: + secretTargetRef: + - parameter: authURL + name: openstack-secret-appcredentials + key: authURL + - parameter: appCredentialID + name: openstack-secret-appcredentials + key: appCredentialID + - parameter: appCredentialSecret + name: openstack-secret-appcredentials + key: appCredentialSecret +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: swift-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + pollingInterval: 10 + cooldownPeriod: 10 + minReplicaCount: 0 + triggers: + - type: openstack-swift + metadata: + swiftURL: http://localhost:8080/v1/AUTH_b161dc518cd24bda84d94d9a0e73fc87 + containerName: my-container + objectCount: "1" + authenticationRef: + name: swift-appcredentials-trigger-authentication +``` \ No newline at end of file diff --git a/content/docs/2.7/scalers/postgresql.md b/content/docs/2.7/scalers/postgresql.md new file mode 100644 index 00000000000..732352022d4 --- /dev/null +++ b/content/docs/2.7/scalers/postgresql.md @@ -0,0 +1,104 @@ ++++ +title = "PostgreSQL" +layout = "scaler" +availability = "v1.2+" +maintainer = "Community" +description = "Scale applications based on a PostgreSQL query." +go_file = "postgresql_scaler" ++++ + +### Trigger Specification + +This specification describes the `postgresql` trigger that scales based on a PostgreSQL query + +The PostgreSQL scaler allows for two connection options: + +A user can offer a full connection string +(often in the form of an environment variable secret) + +- `connectionFromEnv` - PostgreSQL connection string that should point to environment variable with valid value. + +Alternatively, a user can specify individual +arguments (host, userName, password, etc.), and the scaler will form a connection string +internally. + +- `host:` - Service URL to postgresql. Note that you should use a full svc URL as KEDA will need to contact postgresql from a different namespace. +- `userName:` - Username for postgresql user. +- `passwordFromEnv` Password for postgresql user. +- `port` - Postgresql port. +- `dbName` - Postgresql Database name. +- `sslmode` - SSL policy for communicating with database. + +Finally, a user inserts a query that returns the desired value. + +- `query` - What query to poll postgresql with. Query must return an integer. +- `targetQueryValue` - A threshold that is used as `targetAverageValue` in HPA. +- `metricName` - Name to assign to the metric. If not set KEDA will generate a name based on either the connection string if set or the db name. If using more than one trigger it is required that all metricNames be unique. (Optional) + +This is an example of using a full connection string with `AIRFLOW_CONN_AIRFLOW_DB` set as `postgresql://test@localhost`: + +```yaml +triggers: +- type: postgresql + metadata: + connectionFromEnv: AIRFLOW_CONN_AIRFLOW_DB + query: "SELECT ceil(COUNT(*)::decimal / 16) FROM task_instance WHERE state='running' OR state='queued'" + targetQueryValue: 1 + metricName: backlog_process_count #optional. Generated value would be `postgresql-postgresql---test@localhost` +``` + +While this is an example of specifying each parameter: + +```yaml +triggers: +- type: postgresql + metadata: + userName: "kedaUser" + passwordFromEnv: PG_PASSWORD + host: postgres-svc.namespace.cluster.local #use the cluster-wide namespace as KEDA + #lives in a different namespace from your postgres + port: "5432" + dbName: test_db_name + sslmode: disable + query: "SELECT ceil(COUNT(*)::decimal / 16) FROM task_instance WHERE state='running' OR state='queued'" + targetQueryValue: 1 + metricName: backlog_process_count #optional. Generated value would be `postgresql-test_db_name` +``` + +### Authentication Parameters + +You can authenticate by using a password or store the password within the connectionString. + +**Connection String Authentication:** + +- `connection` - Connection string for PostgreSQL database. + +**Password Authentication:** + +- `host` - Service URL to PostgreSQL. Note that you should use a fully qualified URL (including the namespace) as KEDA will need to contact PostgreSQL from a different namespace. +- `userName` - Username for PostgreSQL user. +- `password` Password for configured user to login to PostgreSQL database variables. +- `port` - PostgreSQL port. +- `dbName` - PostgreSQL Database name. +- `sslmode` - SSL policy for communicating with database. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: airflow-worker +spec: + scaleTargetRef: + name: airflow-worker + pollingInterval: 10 # Optional. Default: 30 seconds + cooldownPeriod: 30 # Optional. Default: 300 seconds + maxReplicaCount: 10 # Optional. Default: 100 + triggers: + - type: postgresql + metadata: + connectionFromEnv: AIRFLOW_CONN_AIRFLOW_DB + query: "SELECT ceil(COUNT(*)::decimal / 16) FROM task_instance WHERE state='running' OR state='queued'" + targetQueryValue: 1 +``` diff --git a/content/docs/2.7/scalers/predictkube.md b/content/docs/2.7/scalers/predictkube.md new file mode 100644 index 00000000000..477732a9ce8 --- /dev/null +++ b/content/docs/2.7/scalers/predictkube.md @@ -0,0 +1,111 @@ ++++ +title = "Predictkube" +availability = "v2.6+" +maintainer = "Dysnix" +description = "AI-based predictive scaling based on Prometheus metrics & PredictKube SaaS." +layout = "scaler" +go_file = "predictkube_scaler" ++++ + +PredictKube is an open-source project with the SAAS part consisting of an AI model that requires API to connect to the project for the main functions of predicting and scaling. + +To make our AI model access your data and make a prediction based on it, please use the API key we'll send to your e-mail. + +Review our [Privacy Policy](https://predictkube.com/privacy-policy) to see how your data circulates in and out PredictKube. + +### Trigger Specification + +This specification describes the `predictkube` trigger that scales based on a predicting load based on `prometheus` metrics. + +```yaml +triggers: +- type: predictkube + metadata: + # Required fields: + predictHorizon: "2h" + historyTimeWindow: "7d" + prometheusAddress: http://:9090 + query: sum(rate(http_requests_total{deployment="my-deployment"}[2m])) + queryStep: "2m" + threshold: '100' +``` + +**Parameter list:** + +- `predictHorizon` - Prediction time interval. It is usually equal to the cool-down period of your application. +- `historyTimeWindow` - Time range for which to request metrics from Prometheus. We recommend using minimum 7-14 days time window as historical data. +- `prometheusAddress` - Address of Prometheus server. +- `query` - Predict the query that will yield the value for the scaler to compare against the `threshold`. The query must return a vector/scalar single element response. +- `queryStep` - The maximum time between two slices within the boundaries for QML range query, used in the query. +- `threshold` - Value to start scaling for. + +### Authentication Parameters + +Predictkube Scaler supports one type of authentication - authentication by API key. +Prometheus used in Predictkube Scaler supports all authentication methods that are available in Prometheus Scaler. + +**Auth gateway based authentication:** + +- `apiKey` - API key previously issued for this tenant. You can get your API key by clicking on any **GET API KEY** button on the [website of PredictKube](https://predictkube.com/) + +### Example + +```yaml +--- +apiVersion: v1 +kind: Secret +metadata: + name: predictkube-secrets + namespace: some-namespace +type: Opaque +data: + apiKey: # Required: base64 encoded value of PredictKube apiKey + bearerToken: "BEARER_TOKEN" # Optional: bearer authentication for Prometheus + ca: "CUSTOM_CA_CERT" # Optional: certificate authority file for TLS client authentication for Prometheus +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-predictkube-secret + namespace: some-namespace +spec: + secretTargetRef: + # Required: API key for your predictkube account + - parameter: apiKey + name: predictkube-secrets + key: apiKey + # Optional: might be required if you're using bearer auth for Promethrus + - parameter: bearerToken + name: keda-prom-secret + key: bearerToken + # Optional: might be required if you're using a custom CA for Promethrus + - parameter: ca + name: keda-prom-secret + key: ca +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: predictkube-scaledobject + namespace: some-namespace +spec: + scaleTargetRef: + name: my-deployment + kind: StatefulSet + pollingInterval: 30 + cooldownPeriod: 7200 + minReplicaCount: 3 + maxReplicaCount: 50 + triggers: + - type: predictkube + metadata: + predictHorizon: "2h" + historyTimeWindow: "7d" + prometheusAddress: http://:9090 + query: sum(rate(http_requests_total{deployment="my-deployment"}[2m])) # Note: query must return a vector/scalar single element response + queryStep: "2m" # Note: query step duration for range prometheus queries + threshold: "100" + authModes: "bearer" # might be required if you're using bearer auth for Promethrus + authenticationRef: + name: keda-trigger-auth-predictkube-secret +``` diff --git a/content/docs/2.7/scalers/prometheus.md b/content/docs/2.7/scalers/prometheus.md new file mode 100644 index 00000000000..8cf918c7910 --- /dev/null +++ b/content/docs/2.7/scalers/prometheus.md @@ -0,0 +1,290 @@ ++++ +title = "Prometheus" +layout = "scaler" +availability = "v1.0+" +maintainer = "Community" +description = "Scale applications based on Prometheus." +go_file = "prometheus_scaler" ++++ + +### Trigger Specification + +This specification describes the `prometheus` trigger that scales based on a Prometheus. + +```yaml +triggers: +- type: prometheus + metadata: + # Required fields: + serverAddress: http://:9090 + metricName: http_requests_total # Note: name to identify the metric, generated value would be `prometheus-http_requests_total` + query: sum(rate(http_requests_total{deployment="my-deployment"}[2m])) # Note: query must return a vector/scalar single element response + threshold: '100' + # Optional fields: + namespace: example-namespace # for namespaced queries, eg. Thanos +``` + +**Parameter list:** + +- `serverAddress` - Address of Prometheus. server +- `metricName` - Name to identify the Metric in the external.metrics.k8s.io API. If using more than one trigger it is required that all `metricName`(s) be unique. +- `query` - Query to run. +- `threshold` - Value to start scaling for. +- `namespace` - A namespace that should be used for namespaced queries. These are required by some highly available Prometheus setups, such as [Thanos](https://thanos.io). (Optional) + +### Authentication Parameters + +Prometheus Scaler supports three types of authentication - bearer authentication, basic authentication and TLS authentication. + +You can use `TriggerAuthentication` CRD to configure the authentication. It is possible to specify multiple authentication types i.e. `authModes: "tls,basic"` Specify `authModes` and other trigger parameters along with secret credentials in `TriggerAuthentication` as mentioned below: + +**Bearer authentication:** +- `authModes`: It must contain `bearer` in case of Bearer Authentication. Specify this in trigger configuration. +- `bearerToken`: The token needed for authentication. This is a required field. + +**Basic authentication:** +- `authMode`: It must contain `basic` in case of Basic Authentication. Specify this in trigger configuration. +- `username` - This is a required field. Provide the username to be used for basic authentication. +- `password` - Provide the password to be used for authentication. For convenience, this has been marked optional, because many applications implement basic auth with a username as apikey and password as empty. + +**TLS authentication:** +- `authMode`: It must contain `tls` in case of TLS Authentication. Specify this in trigger configuration. +- `ca` - Certificate authority file for TLS client authentication. +- `cert` - Certificate for client authentication. This is a required field. +- `key` - Key for client authentication. Optional. This is a required field. + +> 💡 **NOTE:**It's also possible to set the CA certificate regardless of the selected `authMode` (also without any authentication). This might be useful if you are using an enterprise CA. + +### Example + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: prometheus-scaledobject + namespace: default +spec: + scaleTargetRef: + name: my-deployment + triggers: + - type: prometheus + metadata: + serverAddress: http://:9090 + metricName: http_requests_total + threshold: '100' + query: sum(rate(http_requests_total{deployment="my-deployment"}[2m])) +``` + +Here is an example of a prometheus scaler with bearer authentication, + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-prom-secret + namespace: default +data: + bearerToken: "BEARER_TOKEN" + ca: "CUSTOM_CA_CERT" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-prom-creds + namespace: default +spec: + secretTargetRef: + - parameter: bearerToken + name: keda-prom-secret + key: bearerToken + # might be required if you're using a custom CA + - parameter: ca + name: keda-prom-secret + key: ca +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: prometheus-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: prometheus + metadata: + serverAddress: http://:9090 + metricName: http_requests_total + threshold: '100' + query: sum(rate(http_requests_total{deployment="my-deployment"}[2m])) + authModes: "bearer" + authenticationRef: + name: keda-prom-creds +``` + +Here is an example of a prometheus scaler with Basic Authentication, define the `Secret` and `TriggerAuthentication` as follows + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-prom-secret + namespace: default +data: + username: "dXNlcm5hbWUK" # Must be base64 + password: "cGFzc3dvcmQK" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-prom-creds + namespace: default +spec: + secretTargetRef: + - parameter: username + name: keda-prom-secret + key: username + - parameter: password + name: keda-prom-secret + key: password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: prometheus-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: metrics-api + metadata: + serverAddress: http://:9090 + metricName: http_requests_total + threshold: '100' + query: sum(rate(http_requests_total{deployment="my-deployment"}[2m])) + authModes: "basic" + authenticationRef: + name: keda-prom-creds +``` + + +Here is an example of a prometheus scaler with TLS Authentication, define the `Secret` and `TriggerAuthentication` as follows + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-prom-secret + namespace: default +data: + cert: "cert" + key: "key" + ca: "ca" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-prom-creds + namespace: default +spec: + secretTargetRef: + - parameter: cert + name: keda-prom-secret + key: cert + - parameter: key + name: keda-prom-secret + key: key + - parameter: ca + name: keda-prom-secret + key: ca +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: prometheus-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: metrics-api + metadata: + serverAddress: http://:9090 + metricName: http_requests_total + threshold: '100' + query: sum(rate(http_requests_total{deployment="my-deployment"}[2m])) + authModes: "tls" + authenticationRef: + name: keda-prom-creds +``` + +Here is an example of a prometheus scaler with TLS and Basic Authentication, define the `Secret` and `TriggerAuthentication` as follows + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-prom-secret + namespace: default +data: + cert: "cert" + key: "key" + ca: "ca" + username: "username" + password: "password" +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-prom-creds + namespace: default +spec: + secretTargetRef: + - parameter: cert + name: keda-prom-secret + key: cert + - parameter: key + name: keda-prom-secret + key: key + - parameter: ca + name: keda-prom-secret + key: ca + - parameter: username + name: keda-prom-secret + key: username + - parameter: password + name: keda-prom-secret + key: password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: prometheus-scaledobject + namespace: keda + labels: + deploymentName: dummy +spec: + maxReplicaCount: 12 + scaleTargetRef: + name: dummy + triggers: + - type: metrics-api + metadata: + serverAddress: http://:9090 + metricName: http_requests_total + threshold: '100' + query: sum(rate(http_requests_total{deployment="my-deployment"}[2m])) + authModes: "tls,basic" + authenticationRef: + name: keda-prom-creds +``` \ No newline at end of file diff --git a/content/docs/2.7/scalers/rabbitmq-queue.md b/content/docs/2.7/scalers/rabbitmq-queue.md new file mode 100644 index 00000000000..80daef38777 --- /dev/null +++ b/content/docs/2.7/scalers/rabbitmq-queue.md @@ -0,0 +1,240 @@ ++++ +title = "RabbitMQ Queue" +layout = "scaler" +availability = "v1.0+" +maintainer = "Microsoft" +description = "Scale applications based on RabbitMQ Queue." +go_file = "rabbitmq_scaler" ++++ + +### Trigger Specification + +This specification describes the `rabbitmq` trigger for RabbitMQ Queue. + +```yaml +triggers: +- type: rabbitmq + metadata: + host: amqp://localhost:5672/vhost # Optional. If not specified, it must be done by using TriggerAuthentication. + protocol: auto # Optional. Specifies protocol to use, either amqp or http, or auto to autodetect based on the `host` value. Default value is auto. + mode: QueueLength # QueueLength or MessageRate + value: "100" # message backlog or publish/sec. target per instance + queueName: testqueue + vhostName: / # Optional. If not specified, use the vhost in the `host` connection string. + # Alternatively, you can use existing environment variables to read configuration from: + # See details in "Parameter list" section + hostFromEnv: RABBITMQ_HOST # Optional. You can use this instead of `host` parameter +``` + +**Parameter list:** + +- `host` - Host of RabbitMQ with format `://:/vhost`. The resolved host should follow a format like `amqp://guest:password@localhost:5672/vhost` or `http://guest:password@localhost:15672/vhost`. When using a username/password consider using `hostFromEnv` or a TriggerAuthentication. +- `queueName` - Name of the queue to read message from. +- `mode` - QueueLength to trigger on number of messages in the queue. MessageRate to trigger on the publish rate into the queue. (Values: `QueueLength`, `MessageRate`) +- `value` - Message backlog or Publish/sec. rate to trigger on. +- `protocol` - Protocol to be used for communication. (Values: `auto`, `http`, `amqp`, Default: `auto`, Optional) +- `vhostName` - Vhost to use for the connection, overrides any vhost set in the connection string from `host`/`hostFromEnv`. (Optional) +- `queueLength` - DEPRECATED! Use `mode: QueueLength` and `value: ##` instead. Target value for queue length passed to the scaler. Example: if one pod can handle 10 messages, set the queue length target to 10. If the actual number of messages in the queue is 30, the scaler scales to 3 pods. Default is 20 unless `publishRate` is specified, in which case `queueLength` is disabled for this trigger. +- `useRegex` - This parameter allows to use regex (in `queueName` parameter) to select queue instead of full name. (Values: `true`, `false`, Default: `false`, Optional, Only applies to hosts that use the `http` protocol) +- `pageSize` - This parameter allows setting page size. (Default: `100`, Optional, Only applies when `useRegex` is `true`) +- `operation` - Operation that will be applied to compute the number of messages in case of `useRegex` enabled. Either `sum` (default),`max`, or `avg`. (Optional) +- `metricName` - Name to assign to the metric. If not set KEDA will generate a name based on the queue name. If using more than one trigger it is required that all metricNames be unique. (Optional) +- `timeout` - Timeout **for this specific trigger**. This value will override the value defined in `KEDA_HTTP_DEFAULT_TIMEOUT`. (Optional, Only applies to hosts that use the `http` protocol) + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `hostFromEnv` - The host and port of the RabbitMQ server, similar to `host`, but reads it from an environment variable on the scale target. + +> 💡 **Note:** `host`/`hostFromEnv` has an optional vhost name after the host slash which will be used to scope API request. + +> 💡 **Note:** When using `host`/`hostFromEnv` or TriggerAuthentication, the supplied password cannot contain special characters. + +> 💡 **Note:** `mode: MessageRate` requires protocol `http`. + +> 💡 **Note:** `useRegex: "true"` requires protocol `http` and ignores unacknowledged messages. + +> ⚠ **Important:** if you have unacknowledged messages and want to have these counted for the scaling to happen, make sure to utilize the `http` REST API interface which allows for these to be counted. + +> ⚠ **Important:** If scaling against both is desired then the `ScaledObject` should have two triggers, one for `mode: QueueLength` and the other for `mode: MessageRate`. HPA will scale based on the largest result considering each of the two triggers independently. + +### Authentication Parameters + +TriggerAuthentication CRD is used to connect and authenticate to RabbitMQ: + +- For AMQP, the URI should look similar to `amqp://guest:password@localhost:5672/vhost`. +- For HTTP, the URI should look similar to `http://guest:password@localhost:15672/vhost`. + +> See the [RabbitMQ Ports](https://www.rabbitmq.com/networking.html#ports) section for more details on how to configure the ports. + +### Example + +#### AMQP protocol: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-rabbitmq-secret +data: + host: # base64 encoded value of format amqp://guest:password@localhost:5672/vhost +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-rabbitmq-conn + namespace: default +spec: + secretTargetRef: + - parameter: host + name: keda-rabbitmq-secret + key: host +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: rabbitmq-scaledobject + namespace: default +spec: + scaleTargetRef: + name: rabbitmq-deployment + triggers: + - type: rabbitmq + metadata: + protocol: amqp + queueName: testqueue + mode: QueueLength + value: "20" + metricName: custom-testqueue #optional. Generated value would be `rabbitmq-custom-testqueue` + authenticationRef: + name: keda-trigger-auth-rabbitmq-conn +``` + +#### HTTP protocol (`QueueLength`): + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-rabbitmq-secret +data: + host: # base64 encoded value of format http://guest:password@localhost:15672/vhost +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-rabbitmq-conn + namespace: default +spec: + secretTargetRef: + - parameter: host + name: keda-rabbitmq-secret + key: host +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: rabbitmq-scaledobject + namespace: default +spec: + scaleTargetRef: + name: rabbitmq-deployment + triggers: + - type: rabbitmq + metadata: + protocol: http + queueName: testqueue + mode: QueueLength + value: "20" + authenticationRef: + name: keda-trigger-auth-rabbitmq-conn +``` + +#### HTTP protocol (`MessageRate` and `QueueLength`): + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-rabbitmq-secret +data: + host: # base64 encoded value of format http://guest:password@localhost:15672/vhost +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-rabbitmq-conn + namespace: default +spec: + secretTargetRef: + - parameter: host + name: keda-rabbitmq-secret + key: host +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: rabbitmq-scaledobject + namespace: default +spec: + scaleTargetRef: + name: rabbitmq-deployment + triggers: + - type: rabbitmq + metadata: + protocol: http + queueName: testqueue + mode: QueueLength + value: "20" + authenticationRef: + name: keda-trigger-auth-rabbitmq-conn + - type: rabbitmq + metadata: + protocol: http + queueName: testqueue + mode: MessageRate + value: "100" + authenticationRef: + name: keda-trigger-auth-rabbitmq-conn +``` + +#### HTTP protocol (`QueueLength`) and using regex (`useRegex`): + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: keda-rabbitmq-secret +data: + host: # base64 encoded value of format http://guest:password@localhost:15672/vhost +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-rabbitmq-conn + namespace: default +spec: + secretTargetRef: + - parameter: host + name: keda-rabbitmq-secret + key: host +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: rabbitmq-scaledobject + namespace: default +spec: + scaleTargetRef: + name: rabbitmq-deployment + triggers: + - type: rabbitmq + metadata: + protocol: http + queueName: ^.*incoming$ + mode: QueueLength + value: "20" + useRegex: "true" + operation: max + authenticationRef: + name: keda-trigger-auth-rabbitmq-conn +``` diff --git a/content/docs/2.7/scalers/redis-cluster-lists.md b/content/docs/2.7/scalers/redis-cluster-lists.md new file mode 100644 index 00000000000..4f515ff09bb --- /dev/null +++ b/content/docs/2.7/scalers/redis-cluster-lists.md @@ -0,0 +1,109 @@ ++++ +title = "Redis Lists (supports Redis Cluster)" +layout = "scaler" +availability = "v2.1+" +maintainer = "Community" +description = "Redis Lists scaler with support for Redis Cluster topology" +go_file = "redis_scaler" ++++ + +### Trigger Specification + +This specification describes the `redis-cluster` trigger that scales based on the length of a list in a Redis Cluster. + +```yaml +triggers: +- type: redis-cluster + metadata: + addresses: localhost:6379 # Comma separated list of the format host:port + usernameFromEnv: REDIS_USERNAME # optional + passwordFromEnv: REDIS_PASSWORD + listName: mylist # Required + listLength: "5" # Required + enableTLS: "false" # optional + # Alternatively, you can use existing environment variables to read configuration from: + # See details in "Parameter list" section + addressesFromEnv: REDIS_ADDRESSES # Optional. You can use this instead of `addresses` parameter +``` + +**Parameter list:** + +- `addresses` - Comma separated list of hosts and ports of the Redis Cluster nodes. +- `hosts` - Comma separated list of hosts of the Redis Cluster nodes. Alternative to `addresses` and requires `ports` to be configured as well. +- `ports` - Comma separated list of corresponding ports for the hosts of the Redis Cluster nodes. Alternative to `addresses` and requires `hosts` to be configured as well. +- `usernameFromEnv` - Environment variable to read the authentication username from to authenticate with the Redis server. +- `passwordFromEnv` - Environment variable to read the authentication password from to authenticate with the Redis server. + - Both the hostname, username and password fields need to be set to the names of the environment variables in the target deployment that contain the host name, username and password respectively. +- `listName` - Name of the Redis List that you want to monitor. +- `listLength` - Average target value to trigger scaling actions. +- `enableTLS` - Allow a connection to a redis queue using tls. (Values: `true`, `false`, Default: `false`, Optional) + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `addressesFromEnv` - The hosts and their respective ports of the Redis Cluster nodes, similar to `addresses`, but reads it from an environment variable on the scale target. +- `hostsFromEnv` - The hosts of the Redis Cluster nodes, similar to `hosts`, but reads it from an environment variable on the scale target. +- `portsFromEnv` - The corresponding ports for the hosts of the Redis Cluster nodes, similar to `ports`, but reads it from an environment variable on the scale target. + +### Authentication Parameters + +You can authenticate by using a password. + +**Connection Authentication:** + +- `addresses` - Comma separated list of host:port format. +- `hosts` - Comma separated list of hostname of the Redis Cluster nodes. If specified, the `ports` should also be specified. +- `ports` - Comma separated list of ports of the Redis Cluster nodes. If specified, the `hosts` should also be specified. + +**Authentication:** + +- `username` - Redis username to authenticate with. +- `password` - Redis password to authenticate with. + +### Example + +Here is an example of how to deploy a scaled object with the `redis-cluster` scale trigger which uses `TriggerAuthentication`. + +You can also provide the `usernameFromEnv` and `passwordFromEnv` on the `ScaledObject` directly. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: votes-db-secret + namespace: my-project +type: Opaque +data: + redis_username: YWRtaW4= + redis_password: YWRtaW4= +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-redis-secret + namespace: my-project +spec: + secretTargetRef: + - parameter: username + name: votes-db-secret + key: redis_username + - parameter: password + name: votes-db-secret + key: redis_password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-scaledobject + namespace: my-project +spec: + scaleTargetRef: + name: votes + triggers: + - type: redis-cluster + metadata: + addresses: node1:6379, node2:6379, node3:6379 + listName: mylist + listLength: "10" + authenticationRef: + name: keda-trigger-auth-redis-secret +``` diff --git a/content/docs/2.7/scalers/redis-cluster-streams.md b/content/docs/2.7/scalers/redis-cluster-streams.md new file mode 100644 index 00000000000..8ebe30b65c5 --- /dev/null +++ b/content/docs/2.7/scalers/redis-cluster-streams.md @@ -0,0 +1,151 @@ ++++ +title = "Redis Streams (supports Redis Cluster)" +layout = "scaler" +availability = "v2.1+" +maintainer = "Community" +description = "Redis Streams scaler with support for Redis Cluster topology" +go_file = "redis_streams_scaler" ++++ + +### Trigger Specification + +Redis 5.0 introduced [Redis Streams](https://redis.io/topics/streams-intro) which is an append-only log data structure. + +One of its features includes [`Consumer Groups`](https://redis.io/topics/streams-intro#consumer-groups), that allows a group of clients to co-operate consuming a different portion of the same stream of messages. + +This specification describes the `redis-cluster-streams` trigger that scales based on the *Pending Entries List* (see [`XPENDING`](https://redis.io/commands/xpending)) for a specific Consumer Group of a Redis Stream and supports Redis Cluster topology. + + +```yaml +triggers: +- type: redis-cluster-streams + metadata: + addresses: localhost:6379 # Required if hosts and ports are not provided. Format - comma separated list of host:port + hosts: localhost # Comma separated lists of hosts. Required if address is not provided + ports: "6379" # Comma separated lists of ports. Required if addresses are not provided and hosts has been provided. + usernameFromEnv: REDIS_USERNAME # optional (can also use authenticationRef) + passwordFromEnv: REDIS_PASSWORD # optional (can also use authenticationRef) + stream: my-stream # Required - name of the Redis Stream + consumerGroup: my-consumer-group # Required - name of consumer group associated with Redis Stream + pendingEntriesCount: "10" # Required - number of entries in the Pending Entries List for the specified consumer group in the Redis Stream + enableTLS: "false" # optional + # Alternatively, you can use existing environment variables to read configuration from: + # See details in "Parameter list" section + addressesFromEnv: REDIS_ADDRESSES # Optional. You can use this instead of `addresses` parameter + hostsFromEnv: REDIS_HOSTS # Optional. You can use this instead of `hosts` parameter + portsFromEnv: REDIS_PORTS # Optional. You can use this instead of `ports` parameter +``` + +**Parameter list:** + +- `addresses` - Comma separated list of hosts and ports of Redis Cluster nodes in the format `host:port` for example `node1:6379, node2:6379, node3:6379`. + +> As an alternative to the `addresses` field, the user can specify `hosts` and `ports` parameters. + +- `hosts` - Comma separated list of hosts of Redis Cluster nodes. + +> It is not required if `addresses` has been provided. + +- `ports`: Comma separated list of ports for corresponding hosts of Redis Cluster nodes. + +> It is only to be used along with the `hosts`/`hostsFromEnv` attribute and not required if `addresses` has been provided. + +- `usernameFromEnv` - Name of the environment variable your deployment uses to get the Redis username. (Optional) +- `passwordFromEnv` - Name of the environment variable your deployment uses to get the Redis password. (Optional) + +- `stream` - Name of the Redis Stream. +- `consumerGroup` - Name of the Consumer group associated with Redis Stream. +- `pendingEntriesCount` - Threshold for the number of `Pending Entries List`. This is the average target value to scale the workload. (Default: `5`, Optional) +- `enableTLS` - Allow a connection to Redis using tls. (Values: `true`, `false`, Default: `false`, Optional) + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `addressesFromEnv` - The hosts and corresponding ports of Redis Cluster nodes, similar to `addresses`, but reads it from an environment variable on the scale target. Name of the environment variable your deployment uses to get the URLs of Redis Cluster nodes. The resolved hosts should follow a format like `node1:6379, node2:6379, node3:6379 ...`. +- `hostsFromEnv` - The hosts of the Redis Cluster nodes, similar to `hosts`, but reads it from an environment variable on the scale target. +- `portsFromEnv` - The corresponding ports for the hosts of Redis Cluster nodes, similar to `ports`, but reads it from an environment variable on the scale target. + +### Authentication Parameters + +The scaler supports two modes of authentication: + +#### Using username/password authentication + +Use the `username` and `password` field in the `metadata` to specify the name of an environment variable that your deployment uses to get the Redis username/password. + +This is usually resolved from a `Secret V1` or a `ConfigMap V1` collections. `env` and `envFrom` are both supported. + +Here is an example: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-streams-scaledobject + namespace: default +spec: + scaleTargetRef: + name: redis-streams-consumer + pollingInterval: 15 + cooldownPeriod: 200 + maxReplicaCount: 25 + minReplicaCount: 1 + triggers: + - type: redis-cluster-streams + metadata: + addressesFromEnv: REDIS_ADDRESSES + usernameFromEnv: REDIS_USERNAME # name of the environment variable in the Deployment + passwordFromEnv: REDIS_PASSWORD # name of the environment variable in the Deployment + stream: my-stream + consumerGroup: consumer-group-1 + pendingEntriesCount: "10" +``` + +#### Using `TriggerAuthentication` + +You can use `TriggerAuthentication` CRD to configure the authentication. For example: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: redis-streams-auth +type: Opaque +data: + redis_username: + redis_password: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-redis-stream-triggerauth +spec: + secretTargetRef: + - parameter: username + name: redis-streams-auth # name of the Secret + key: redis_username # name of the key in the Secret + - parameter: password + name: redis-streams-auth # name of the Secret + key: redis_password # name of the key in the Secret +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-streams-scaledobject + namespace: default +spec: + scaleTargetRef: + name: redis-streams-consumer + pollingInterval: 15 + cooldownPeriod: 200 + maxReplicaCount: 25 + minReplicaCount: 1 + triggers: + - type: redis-cluster-streams + metadata: + address: node1:6379, node2:6379, node3:6379 + stream: my-stream + consumerGroup: consumer-group-1 + pendingEntriesCount: "10" + authenticationRef: + name: keda-redis-stream-triggerauth # name of the TriggerAuthentication resource +``` diff --git a/content/docs/2.7/scalers/redis-lists.md b/content/docs/2.7/scalers/redis-lists.md new file mode 100644 index 00000000000..8642e8902e9 --- /dev/null +++ b/content/docs/2.7/scalers/redis-lists.md @@ -0,0 +1,111 @@ ++++ +title = "Redis Lists" +layout = "scaler" +availability = "v1.0+" +maintainer = "Community" +description = "Scale applications based on Redis Lists." +go_file = "redis_scaler" ++++ + +### Trigger Specification + +This specification describes the `redis` trigger that scales based on the length of a list in Redis. + +```yaml +triggers: +- type: redis + metadata: + address: localhost:6379 # Format must be host:port + usernameFromEnv: REDIS_USERNAME # optional + passwordFromEnv: REDIS_PASSWORD + listName: mylist # Required + listLength: "5" # Required + enableTLS: "false" # optional + databaseIndex: "0" # optional + # Alternatively, you can use existing environment variables to read configuration from: + # See details in "Parameter list" section + addressFromEnv: REDIS_HOST # Optional. You can use this instead of `address` parameter +``` + +**Parameter list:** + +- `address` - The host and port of the Redis server. +- `host` - The host of the Redis server. Alternative to `address` and requires `port` to be configured as well. +- `port` - The port of the Redis server. Alternative to `address` and requires `host` to be configured as well. +- `usernameFromEnv` - Environment variable to read the authentication username from to authenticate with the Redis server. +- `passwordFromEnv` - Environment variable to read the authentication password from to authenticate with the Redis server. + - Both the hostname, username and password fields need to be set to the names of the environment variables in the target deployment that contain the host name, username and password respectively. +- `listName` - Name of the Redis List that you want to monitor. +- `listLength` - Average target value to trigger scaling actions. +- `enableTLS` - Allow a connection to a redis queue using tls. (Values: `true`, `false`, Default: `false`, Optional) +- `databaseIndex` - Index of Redis database to use. If not specified, the default value is 0. + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `addressFromEnv` - The host and port of the Redis server, similar to `address`, but reads it from an environment variable on the scale target. +- `hostFromEnv` - The host of the Redis server, similar to `host`, but reads it from an environment variable on the scale target. +- `portFromEnv` - The port of the Redis server, similar to `port`, but reads it from an environment variable on the scale target. + +### Authentication Parameters + +You can authenticate by using a username (optional) and password. + +**Connection Authentication:** + +- `address` - The hostname and port for the Redis server (host:port format). +- `host` - The hostname of the Redis server. If specified, the `port` should also be specified. +- `port` - The port of the Redis server. If specified, the `host` should also be specified. + +**Authentication:** + +- `username` - Redis username to authenticate with. +- `password` - Redis password to authenticate with. + +### Example + +Here is an example of how to deploy a scaled object with the `redis` scale trigger which uses `TriggerAuthentication`. + +You can also provide the `usernameFromEnv` and `passwordFromEnv` on the `ScaledObject` directly. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: votes-db-secret + namespace: my-project +type: Opaque +data: + redis_username: YWRtaW4= + redis_password: YWRtaW4= +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-redis-secret + namespace: my-project +spec: + secretTargetRef: + - parameter: username + name: votes-db-secret + key: redis_username + - parameter: password + name: votes-db-secret + key: redis_password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-scaledobject + namespace: my-project +spec: + scaleTargetRef: + name: votes + triggers: + - type: redis + metadata: + address: localhost:6379 + listName: mylist + listLength: "10" + authenticationRef: + name: keda-trigger-auth-redis-secret +``` diff --git a/content/docs/2.7/scalers/redis-sentinel-lists.md b/content/docs/2.7/scalers/redis-sentinel-lists.md new file mode 100644 index 00000000000..cf1b29712c3 --- /dev/null +++ b/content/docs/2.7/scalers/redis-sentinel-lists.md @@ -0,0 +1,124 @@ ++++ +title = "Redis Lists (supports Redis Sentinel)" +layout = "scaler" +availability = "v2.5+" +maintainer = "Community" +description = "Redis Lists scaler with support for Redis Sentinel topology" +go_file = "redis_scaler" ++++ + +### Trigger Specification + +This specification describes the `redis-sentinel` trigger that scales based on the length of a list in a Redis Sentinel setup. + +```yaml +triggers: +- type: redis-sentinel + metadata: + addresses: localhost:26379 # Comma separated list of the format host:port + usernameFromEnv: REDIS_USERNAME # optional + passwordFromEnv: REDIS_PASSWORD # optional + sentinelUsernameFromEnv: REDIS_SENTINEL_USERNAME # optional + sentinelPasswordFromEnv: REDIS_SENTINEL_PASSWORD # optional + sentinelMasterFromEnv: REDIS_SENTINEL_MASTER # optional + listName: mylist # Required + listLength: "5" # Required + enableTLS: "false" # optional + # Alternatively, you can use existing environment variables to read configuration from: + # See details in "Parameter list" section + addressesFromEnv: REDIS_ADDRESSES # Optional. You can use this instead of `addresses` parameter +``` + +**Parameter list:** + +- `addresses` - Comma separated list of hosts and ports of the Redis Sentinel nodes. +- `hosts` - Comma separated list of hosts of the Redis Sentinel nodes. Alternative to `addresses` and requires `ports` to be configured as well. +- `ports` - Comma separated list of corresponding ports for the hosts of the Redis Sentinel nodes. Alternative to `addresses` and requires `hosts` to be configured as well. +- `usernameFromEnv` - Environment variable to read the authentication username from to authenticate with the Redis server. +- `passwordFromEnv` - Environment variable to read the authentication password from to authenticate with the Redis server. + - Both the hostname, username and password fields need to be set to the names of the environment variables in the target deployment that contain the host name, username and password respectively. +- `sentinelUsernameFromEnv` - Environment variable to read the authentication username from to authenticate with the Redis Sentinel server. +- `sentinelPasswordFromEnv` - Environment variable to read the authentication password from to authenticate with the Redis Sentinel server. +- `sentinelMaster` - The name of the master in Sentinel to get the Redis server address for. +- `listName` - Name of the Redis List that you want to monitor. +- `listLength` - Average target value to trigger scaling actions. +- `enableTLS` - Allow a connection to a redis queue using tls. (Values: `true`, `false`, Default: `false`, Optional) + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `addressesFromEnv` - The hosts and their respective ports of the Redis Sentinel nodes, similar to `addresses`, but reads it from an environment variable on the scale target. +- `hostsFromEnv` - The hosts of the Redis Sentinel nodes, similar to `hosts`, but reads it from an environment variable on the scale target. +- `portsFromEnv` - The corresponding ports for the hosts of the Redis Sentinel nodes, similar to `ports`, but reads it from an environment variable on the scale target. +- `sentinelMasterFromEnv` - The name of the master in Sentinel to get the Redis server address for, similar to `sentinelMaster`, but reads it from an environment variable on the scale target. + +### Authentication Parameters + +You can authenticate by using a password. + +**Connection Authentication:** + +- `addresses` - Comma separated list of host:port format. +- `hosts` - Comma separated list of hostname of the Redis Sentinel nodes. If specified, the `ports` should also be specified. +- `ports` - Comma separated list of ports of the Redis Sentinel nodes. If specified, the `hosts` should also be specified. +- `sentinelMaster` - The name of the master in Sentinel to get the Redis server address for. + +**Authentication:** + +- `username` - Redis username to authenticate with. +- `password` - Redis password to authenticate with. + +**Sentinel Authentication:** + +- `sentinelUsername` - Redis Sentinel username to authenticate with. +- `sentinelPassword` - Redis Sentinel password to authenticate with. + + +### Example + +Here is an example of how to deploy a scaled object with the `redis-sentinel` scale trigger which uses `TriggerAuthentication`. + +You can also provide the `usernameFromEnv` and `passwordFromEnv` on the `ScaledObject` directly. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: votes-db-secret + namespace: my-project +type: Opaque +data: + redis_username: YWRtaW4= + redis_password: YWRtaW4= +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-trigger-auth-redis-secret + namespace: my-project +spec: + secretTargetRef: + - parameter: username + name: votes-db-secret + key: redis_username + - parameter: password + name: votes-db-secret + key: redis_password +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-scaledobject + namespace: my-project +spec: + scaleTargetRef: + name: votes + triggers: + - type: redis-sentinel + metadata: + addresses: node1:26379, node2:26379, node3:26379 + listName: mylist + listLength: "10" + sentinelMaster: "mymaster" + authenticationRef: + name: keda-trigger-auth-redis-secret +``` diff --git a/content/docs/2.7/scalers/redis-sentinel-streams.md b/content/docs/2.7/scalers/redis-sentinel-streams.md new file mode 100644 index 00000000000..514bd728dc9 --- /dev/null +++ b/content/docs/2.7/scalers/redis-sentinel-streams.md @@ -0,0 +1,158 @@ ++++ +title = "Redis Streams (supports Redis Sentinel)" +layout = "scaler" +availability = "v2.5+" +maintainer = "Community" +description = "Redis Streams scaler with support for Redis Sentinel topology" +go_file = "redis_streams_scaler" ++++ + +### Trigger Specification + +Redis 5.0 introduced [Redis Streams](https://redis.io/topics/streams-intro) which is an append-only log data structure. + +One of its features includes [`Consumer Groups`](https://redis.io/topics/streams-intro#consumer-groups), that allows a group of clients to co-operate consuming a different portion of the same stream of messages. + +This specification describes the `redis-sentinel-streams` trigger that scales based on the *Pending Entries List* (see [`XPENDING`](https://redis.io/commands/xpending)) for a specific Consumer Group of a Redis Stream and supports a Redis Sentinel setup. + + +```yaml +triggers: +- type: redis-sentinel-streams + metadata: + addresses: localhost:26379 # Required if hosts and ports are not provided. Format - comma separated list of host:port + hosts: localhost # Comma separated lists of hosts. Required if address is not provided + ports: "26379" # Comma separated lists of ports. Required if addresses are not provided and hosts has been provided. + usernameFromEnv: REDIS_USERNAME # optional (can also use authenticationRef) + passwordFromEnv: REDIS_PASSWORD # optional (can also use authenticationRef) + stream: my-stream # Required - name of the Redis Stream + consumerGroup: my-consumer-group # Required - name of consumer group associated with Redis Stream + pendingEntriesCount: "10" # Required - number of entries in the Pending Entries List for the specified consumer group in the Redis Stream + enableTLS: "false" # optional + # Alternatively, you can use existing environment variables to read configuration from: + # See details in "Parameter list" section + addressesFromEnv: REDIS_ADDRESSES # Optional. You can use this instead of `addresses` parameter + hostsFromEnv: REDIS_HOSTS # Optional. You can use this instead of `hosts` parameter + portsFromEnv: REDIS_PORTS # Optional. You can use this instead of `ports` parameter +``` + +**Parameter list:** + +- `addresses` - Comma separated list of hosts and ports of Redis Sentinel nodes in the format `host:port` for example `node1:26379, node2:26379, node3:26379`. + +> As an alternative to the `addresses` field, the user can specify `hosts` and `ports` parameters. + +- `hosts` - Comma separated list of hosts of Redis Sentinel nodes. + +> It is not required if `addresses` has been provided. + +- `ports`: Comma separated list of ports for corresponding hosts of Redis Sentinel nodes. + +> It is only to be used along with the `hosts`/`hostsFromEnv` attribute and not required if `addresses` has been provided. + +- `usernameFromEnv` - Name of the environment variable your deployment uses to get the Redis username. (Optional) +- `passwordFromEnv` - Name of the environment variable your deployment uses to get the Redis password. (Optional) + +- `sentinelUsernameFromEnv` - Name of the environment variable your deployment uses to get the Redis Sentinel username. (Optional) +- `sentinelPasswordFromEnv` - Name of the environment variable your deployment uses to get the Redis Sentinel password. (Optional) + +- `sentinelMaster` - The name of the master in Sentinel to get the Redis server address for. +- `stream` - Name of the Redis Stream. +- `consumerGroup` - Name of the Consumer group associated with Redis Stream. +- `pendingEntriesCount` - Threshold for the number of `Pending Entries List`. This is the average target value to scale the workload. (Default: `5`, Optional) +- `enableTLS` - Allow a connection to Redis using tls. (Values: `true`, `false`, Default: `false`, Optional) + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `addressesFromEnv` - The hosts and corresponding ports of Redis Sentinel nodes, similar to `addresses`, but reads it from an environment variable on the scale target. Name of the environment variable your deployment uses to get the URLs of Redis Sentinel nodes. The resolved hosts should follow a format like `node1:26379, node2:26379, node3:26379 ...`. +- `hostsFromEnv` - The hosts of the Redis Sentinel nodes, similar to `hosts`, but reads it from an environment variable on the scale target. +- `portsFromEnv` - The corresponding ports for the hosts of Redis Sentinel nodes, similar to `ports`, but reads it from an environment variable on the scale target. +- `sentinelMasterFromEnv` - The name of the master in Sentinel to get the Redis server address for, similar to `sentinelMaster`, but reads it from an environment variable on the scale target. + +### Authentication Parameters + +The scaler supports two modes of authentication: + +#### Using username/password authentication + +Use the `username` and `password` field in the `metadata` to specify the name of an environment variable that your deployment uses to get the Redis username/password. + +This is usually resolved from a `Secret V1` or a `ConfigMap V1` collections. `env` and `envFrom` are both supported. + +Here is an example: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-streams-scaledobject + namespace: default +spec: + scaleTargetRef: + name: redis-streams-consumer + pollingInterval: 15 + cooldownPeriod: 200 + maxReplicaCount: 25 + minReplicaCount: 1 + triggers: + - type: redis-sentinel-streams + metadata: + addressesFromEnv: REDIS_ADDRESSES + usernameFromEnv: REDIS_USERNAME # name of the environment variable in the Deployment + passwordFromEnv: REDIS_PASSWORD # name of the environment variable in the Deployment + stream: my-stream + consumerGroup: consumer-group-1 + pendingEntriesCount: "10" + sentinelMaster: "mymaster" +``` + +#### Using `TriggerAuthentication` + +You can use `TriggerAuthentication` CRD to configure the authentication. For example: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: redis-streams-auth +type: Opaque +data: + redis_username: + redis_password: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-redis-stream-triggerauth +spec: + secretTargetRef: + - parameter: username + name: redis-streams-auth # name of the Secret + key: redis_username # name of the key in the Secret + - parameter: password + name: redis-streams-auth # name of the Secret + key: redis_password # name of the key in the Secret +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-streams-scaledobject + namespace: default +spec: + scaleTargetRef: + name: redis-streams-consumer + pollingInterval: 15 + cooldownPeriod: 200 + maxReplicaCount: 25 + minReplicaCount: 1 + triggers: + - type: redis-sentinel-streams + metadata: + address: node1:26379, node2:26379, node3:26379 + stream: my-stream + consumerGroup: consumer-group-1 + pendingEntriesCount: "10" + sentinelMaster: "mymaster" + authenticationRef: + name: keda-redis-stream-triggerauth # name of the TriggerAuthentication resource +``` diff --git a/content/docs/2.7/scalers/redis-streams.md b/content/docs/2.7/scalers/redis-streams.md new file mode 100644 index 00000000000..db20552cf2e --- /dev/null +++ b/content/docs/2.7/scalers/redis-streams.md @@ -0,0 +1,153 @@ ++++ +title = "Redis Streams" +layout = "scaler" +availability = "v1.5+" +maintainer = "Community" +description = "Scale applications based on Redis Streams." +go_file = "redis_streams_scaler" ++++ + +### Trigger Specification + +Redis 5.0 introduced [Redis Streams](https://redis.io/topics/streams-intro) which is an append-only log data structure. + +One of its features includes [`Consumer Groups`](https://redis.io/topics/streams-intro#consumer-groups), that allows a group of clients to co-operate consuming a different portion of the same stream of messages. + +This specification describes the `redis-streams` trigger that scales based on the *Pending Entries List* (see [`XPENDING`](https://redis.io/commands/xpending)) for a specific Consumer Group of a Redis Stream. + + +```yaml +triggers: +- type: redis-streams + metadata: + address: localhost:6379 # Required if host and port are not provided. Format - host:port + host: localhost # Required if address is not provided + port: "6379" # Required if address is not provided and host has been provided. + usernameFromEnv: REDIS_USERNAME # optional (can also use authenticationRef) + passwordFromEnv: REDIS_PASSWORD # optional (can also use authenticationRef) + stream: my-stream # Required - name of the Redis Stream + consumerGroup: my-consumer-group # Required - name of consumer group associated with Redis Stream + pendingEntriesCount: "10" # Required - number of entries in the Pending Entries List for the specified consumer group in the Redis Stream + enableTLS: "false" # optional + databaseIndex: "0" # optional + # Alternatively, you can use existing environment variables to read configuration from: + # See details in "Parameter list" section + addressFromEnv: REDIS_ADDRESS # Optional. You can use this instead of `address` parameter + hostFromEnv: REDIS_HOST # Optional. You can use this instead of `host` parameter + portFromEnv: REDIS_PORT # Optional. You can use this instead of `port` parameter +``` + +**Parameter list:** + +- `address` - The host and port of the Redis server in the format `host:port`, for example `my-redis:6379`. + +> As an alternative to the `address` field, the user can specify `host` and `port` parameters. + +- `host` - The host of the Redis server. + +> It is not required if `address` has been provided. + +- `port` - The port of the Redis server. + +> It is only to be used along with the `host`/`hostFromEnv` attribute and not required if `address` has been provided. + +- `usernameFromEnv` - Name of the environment variable your deployment uses to get the Redis username. (Optional) +- `passwordFromEnv` - Name of the environment variable your deployment uses to get the Redis password. (Optional) + +- `stream` - Name of the Redis Stream. +- `consumerGroup` - Name of the Consumer group associated with Redis Stream. +- `pendingEntriesCount` - Threshold for the number of `Pending Entries List`. This is the average target value to scale the workload. (Default: `5`, Optional) +- `databaseIndex` - The Redis database index. Defaults to `0` if not specified. +- `enableTLS` - Allow a connection to Redis using tls. (Values: `true`, `false`, Default: `false`, Optional) + +Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables: + +- `addressFromEnv` - The host and port of the Redis server, similar to `address`, but reads it from an environment variable on the scale target. Name of the environment variable your deployment uses to get the Redis server URL. The resolved host should follow a format like `my-redis:6379`. +- `hostFromEnv` - The host of the Redis server, similar to `host`, but reads it from an environment variable on the scale target. +- `portFromEnv` - The port of the Redis server, similar to `port`, but reads it from an environment variable on the scale target. + +### Authentication Parameters + +The scaler supports two modes of authentication: + +#### Using username/password authentication + +Use the `username` and `password` field in the `metadata` to specify the name of an environment variable that your deployment uses to get the Redis username/password. + +This is usually resolved from a `Secret V1` or a `ConfigMap V1` collections. `env` and `envFrom` are both supported. + +Here is an example: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-streams-scaledobject + namespace: default +spec: + scaleTargetRef: + name: redis-streams-consumer + pollingInterval: 15 + cooldownPeriod: 200 + maxReplicaCount: 25 + minReplicaCount: 1 + triggers: + - type: redis-streams + metadata: + addressFromEnv: REDIS_HOST + usernameFromEnv: REDIS_USERNAME # name of the environment variable in the Deployment + passwordFromEnv: REDIS_PASSWORD # name of the environment variable in the Deployment + stream: my-stream + consumerGroup: consumer-group-1 + pendingEntriesCount: "10" +``` + +#### Using `TriggerAuthentication` + +You can use `TriggerAuthentication` CRD to configure the authentication. For example: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: redis-streams-auth +type: Opaque +data: + redis_username: + redis_password: +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: keda-redis-stream-triggerauth +spec: + secretTargetRef: + - parameter: username + name: redis-streams-auth # name of the Secret + key: redis_username # name of the key in the Secret + - parameter: password + name: redis-streams-auth # name of the Secret + key: redis_password # name of the key in the Secret +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: redis-streams-scaledobject + namespace: default +spec: + scaleTargetRef: + name: redis-streams-consumer + pollingInterval: 15 + cooldownPeriod: 200 + maxReplicaCount: 25 + minReplicaCount: 1 + triggers: + - type: redis-streams + metadata: + address: localhost:6379 + stream: my-stream + consumerGroup: consumer-group-1 + pendingEntriesCount: "10" + authenticationRef: + name: keda-redis-stream-triggerauth # name of the TriggerAuthentication resource +``` diff --git a/content/docs/2.7/scalers/selenium-grid-scaler.md b/content/docs/2.7/scalers/selenium-grid-scaler.md new file mode 100644 index 00000000000..71c30020f76 --- /dev/null +++ b/content/docs/2.7/scalers/selenium-grid-scaler.md @@ -0,0 +1,121 @@ ++++ +title = "Selenium Grid Scaler" +availability = "v2.4+" +maintainer = "Volvo Cars" +description = "Scales Selenium browser nodes based on number of requests waiting in session queue" +layout = "scaler" +go_file = "selenium_grid_scaler" ++++ + +### Trigger Specification + +This specification describes the `selenium-grid` trigger that scales browser nodes based on number of requests in session queue. + +The scaler creates one browser node per pending request in session queue. You will have to create one trigger per browser capability that you would like to support in your Selenium Grid. + +The below is an example trigger configuration for chrome node. + +```yaml +triggers: + - type: selenium-grid + metadata: + url: 'http://selenium-hub:4444/graphql' # Required + browserName: 'chrome' # Required + browserVersion: '91.0' # Optional. Only required when supporting multiple versions of browser in your Selenium Grid. + unsafeSsl : 'true' # Optional +``` + +**Parameter list:** + +- `url` - Graphql url of your Selenium Grid. Refer to the Selenium Grid's documentation [here](https://www.selenium.dev/documentation/en/grid/grid_4/graphql_support/) to for more info. +- `browserName` - Name of browser that usually gets passed in the browser capability. Refer to the [Selenium Grid's](https://www.selenium.dev/documentation/en/getting_started_with_webdriver/browsers/) and [WebdriverIO's](https://webdriver.io/docs/options/#capabilities) documentation for more info. +- `browserVersion` - Version of browser that usually gets passed in the browser capability. Refer to the [Selenium Grid's](https://www.selenium.dev/documentation/en/getting_started_with_webdriver/browsers/) and [WebdriverIO's](https://webdriver.io/docs/options/#capabilities) documentation for more info. (Optional) +- `unsafeSsl` - Skip certificate validation when connecting over HTTPS. (Values: `true`, `false`, Default: `false`, Optional) + +### Example + +Here is a full example of scaled object definition using Selenium Grid trigger: + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: selenium-grid-chrome-scaledobject + namespace: keda + labels: + deploymentName: selenium-chrome-node +spec: + maxReplicaCount: 8 + scaleTargetRef: + name: selenium-chrome-node + triggers: + - type: selenium-grid + metadata: + url: 'http://selenium-hub:4444/graphql' + browserName: 'chrome' +``` + +The above example will create Chrome browser nodes equal to the requests pending in session queue for Chrome browser. + +Similary for Firefox + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: selenium-grid-firefox-scaledobject + namespace: keda + labels: + deploymentName: selenium-firefox-node +spec: + maxReplicaCount: 8 + scaleTargetRef: + name: selenium-firefox-node + triggers: + - type: selenium-grid + metadata: + url: 'http://selenium-hub:4444/graphql' + browserName: 'firefox' +``` + +If you are supporting multiple versions of browser capability in your Selenium Grid, You should create one scaler for every browser version and pass the `browserVersion` in the metadata. + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: selenium-grid-chrome-91-scaledobject + namespace: keda + labels: + deploymentName: selenium-chrome-node-91 +spec: + maxReplicaCount: 8 + scaleTargetRef: + name: selenium-chrome-node-91 + triggers: + - type: selenium-grid + metadata: + url: 'http://selenium-hub:4444/graphql' + browserName: 'chrome' + browserVersion: '91.0' +``` + +```yaml +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: selenium-grid-chrome-90-scaledobject + namespace: keda + labels: + deploymentName: selenium-chrome-node-90 +spec: + maxReplicaCount: 8 + scaleTargetRef: + name: selenium-chrome-node-90 + triggers: + - type: selenium-grid + metadata: + url: 'http://selenium-hub:4444/graphql' + browserName: 'chrome' + browserVersion: '90.0' +``` \ No newline at end of file diff --git a/content/docs/2.7/scalers/solace.md b/content/docs/2.7/scalers/solace.md new file mode 100644 index 00000000000..8b4d85168d7 --- /dev/null +++ b/content/docs/2.7/scalers/solace.md @@ -0,0 +1,105 @@ ++++ +title = "Solace PubSub+ Event Broker" +availability = "2.4+" +maintainer = "Community" +description = "Scale applications based on Solace PubSub+ Event Broker Queues" +layout = "scaler" +go_file = "solace_scaler" ++++ + +### Trigger Specification +This specification describes the `solace-event-queue` trigger that scales based on a Solace PubSub+ Event Broker queue. + +```yaml +triggers: +- type: solace-event-queue + metadata: + solaceSempBaseURL: http://solace_broker:8080 + messageVpn: message-vpn + queueName: queue_name + messageCountTarget: '100' + messageSpoolUsageTarget: '100' ### Megabytes (MB) + username: semp-user + password: semp-pwd + usernameFromEnv: ENV_VAR_USER + passwordFromEnv: ENV_VAR_PWD +``` + +**Parameter list:** +- `solaceSempBaseURL` - Solace SEMP Endpoint in format: `://:`. +- `messageVpn` - Message VPN hosted on the Solace broker. +- `queueName` - Message Queue to be monitored. +- `messageCountTarget` - The target number of messages manageable by a pod. The scaler will cause the replicas to increase if the queue message backlog is greater than the target value per active replica. +- `messageSpoolUsageTarget` - Integer value expressed in Megabytes (MB). The target spool usage manageable by a pod. The scaler will cause the replicas to increase if the queue spool usage is greater than the target value per active replica. +- `username` - User account with access to Solace SEMP RESTful endpoint. +- `password` - Password for the user account. +- `usernameFromEnv` - Environment variable set with SEMP user account. +- `passwordFromEnv` - Environment variable set with password for the user account. + +**Parameter Requirements:** +- Parameters resolving the target queue are all **required:** `solaceSempBaseURL`, `messageVpn`, `queueName` +- **At least** one of `messageCountTarget` or `messageSpoolUsageTarget` is **required.** If both values are present, the metric value resulting in the higher desired replicas will be used. (Standard KEDA/HPA behavior) +- The Solace PubSub+ Scaler polls the Solace SEMP REST API to monitor target queues. Currently, the scaler supports basic authentication. `username` and `password` are **required** for the `solace-event-queue` trigger to function. These values may be set directly in the trigger metadata or using a TriggerAuthentication record. See [Authentication Parameters](#authentication-parameters) below. Alternatively, credentials may be passed from environment variables identified by `usernameFromEnv` and `passwordFromEnv`. + +### Authentication Parameters +You can use TriggerAuthentication CRD to configure the username and password to connect to the management endpoint. + +**Username and Password based authentication:** +- `username` - Required. The username to use to connect to the Solace PubSub+ Event Broker's SEMP endpoint. +- `password` - Required. The password to use to connect to the Solace PubSub+ Event Broker's SEMP endpoint. + +### Example +The objects in the example below are declared in `namespace=solace`. It is not required to do so. If you do define a namespace for the configuration objects, then they should all be delcared in the same namespace. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: solace-secret + namespace: solace + labels: + app: solace-consumer +type: Opaque +data: + SEMP_USER: YWRtaW4= + SEMP_PASSWORD: S2VkYUxhYkFkbWluUHdkMQ== +--- +apiVersion: keda.sh/v1alpha1 +kind: ScaledObject +metadata: + name: solace-scaled-object + namespace: solace +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: solace-consumer + pollingInterval: 20 + cooldownPeriod: 60 + minReplicaCount: 0 + maxReplicaCount: 10 + triggers: + - type: solace-event-queue + metadata: + solaceSempBaseURL: http://broker-pubsubplus.solace.svc.cluster.local:8080 + messageVpn: test_vpn + queueName: SCALED_CONSUMER_QUEUE1 + messageCountTarget: '50' + messageSpoolUsageTarget: '100000' + authenticationRef: + name: solace-trigger-auth +--- +apiVersion: keda.sh/v1alpha1 +kind: TriggerAuthentication +metadata: + name: solace-trigger-auth + namespace: solace +spec: + secretTargetRef: + - parameter: username + name: solace-secret + key: SEMP_USER + - parameter: password + name: solace-secret + key: SEMP_PASSWORD +``` diff --git a/content/docs/2.7/troubleshooting.md b/content/docs/2.7/troubleshooting.md new file mode 100644 index 00000000000..9bc0b05b1fb --- /dev/null +++ b/content/docs/2.7/troubleshooting.md @@ -0,0 +1,6 @@ ++++ +title = "Troubleshooting" +description = "How to address commonly encountered KEDA issues" ++++ + +{{< troubleshooting >}}