support cron + metric based hpa

[juliantaylor]

The cron hpa is useful for some cases but it lacks the ability to scale deployments higher than the configured target replica count based on metrics like cpu and memory usage.
Using it together with the k8s horizontalpodautoscaler which does that probably produces conflicts as two operators update the same spec.

The purely metric based hpa can be insufficient when the scaling is to slow to react to predictable usage peaks.

To solve this there are two ways:
write your own custom time based metrics and munge them with the normal cpu and memory metrics to get them to scale preemptively.
This requires implementing this munging and have a cluster which support custom metrics to begin with.

The second much easier approach which imo could be added to this project is to combine both hpa objects the cron object and the horizontalautoscaler object into one object.
So you define your metrics on which to autoscale and define time periods where there must be a minimum amount of replicas available.
That way you get the effect of cronhpa and retain the metric based autoscaling should the load be unexpectedly high.

This could be implemented by this operator simply creating and managing the normal k8s autoscaler objects based on its own crd.
So you still just deploy the combined cronhpa object and the operator will create at the right times the right horizontalautoscaler object with an timescale appropriate minimum and maximum replica spec.

[hex108]

Thanks @juliantaylor for the suggestions! Sorry for later reply.

Yes, we plan to support it in the second way you suggested. It would be very appreciative if you are interested in it.