Provide capability to extend scalers

[tomkerkhove]

In certain scenarios it would be interesting to react based on what a certain scaler is doing.

For example; when processing a Service Bus/RabbitMQ queue but the queue does not exist it would be nice if we could react to that.

Proposal

We should provide the capability to opt-in for events which are emitted by a given scaler. With that, we could trigger a process to react. (Relates to #479)

Alternative

Alternative is to be able to define a job to run as part of the metadata, but personally I think it's best to decouple this from KEDA and provide the info but leave it up to others to handle it.

Other information

This came up on #KEDA in Kubernetes Slack for Tomek Święcicki his scenario.

[tomislater]

I can add some info about scenarios and how we have handled some of them.

Scenario 1

Your consumer/worker (deployment) creates a queue in RabbitMQ. If you create Deployment and ScaledObject in the same time, and for some reason your pod (which creates a queue) takes some time to run, KEDA can be faster and scale your deployment down and thus you will never be able to create a queue (if minReplicaCount is set to 0).

Solutions:

1. Set minReplicaCount to 1, but if you have many workers/consumers and you need them only for a while (when something is on queue) there is no point to set this to 1; you just want to scale them down
2. Create ScaledObject after Deployment. We use Helm, so we can just create ScaledObject in post-install,post-upgrade hooks. And, it works... most of the workers/consumers are up and ready at the time, but there is another scenario 💃

Scenario 2

You have ScaledObject with minReplicaCount set to 0, your Deployment (worker/consumer which creates a queue) has 0 replicas (let's say there was nothing on queue, so KEDA had scaled it down) and for some reason, there is no queue in RabbitMQ (someone has deleted disk, queue was not durable, someone has set TTL on queue, whatever).
With such setup, KEDA will never scale up your deployment because there is no queue in RabbitMQ and it will never be, because a queue is created (which is not a rare case) by your worker/consumer.

Solutions:

1. DITTO, set minReplicaCount to 1
2. Remove ScaledObject objects and scale Deployment to 1 replicas. But, it is not a solution at all.

---

It would be great to tell KEDA what should be done if there is an error. In one case it look okay to scale your deployment to 1 if there are such error (no queue in rabbit), but in another, when something is wrong with backend (RabbitMQ, Redis, etc. (cannot create a disk for instance)) such action does not make any sense as @tomkerkhove has pointed out. But, it is another problem.

[redyamsirisha]

how we can trigger a single workflow after all other workflows are completed in github actions?

[JorTurFer]

@tomislater , could this be solved using fallback feature?

[tomislater]

I think so 🤔