[Task]: Reduce SPS Operational Costs

[LucaCinquini]

In order to reduce SPS cost, it is suggested we do the following changes:

o Run integration tests every few days, instead of every day
o Do not run the most expensive integration test, which is the L1toL2 end-to-end
o Reduce the retention period of database snapshots from 1 month to 1 week

[LucaCinquini]

Other possible changes to reduce costs:

o Set the database instance_class to db.m5d.large instead of db.m5d.2xlarge
(see terraform-unity/modules/terraform-unity-sps-database/main.tf - it should be set as part of the helm values differently in values.tmpl.yaml or values_high_load.tmpl.yaml)

o Reduce the number of control nodes, keep a higher number for high load deployments? Optimally, all Airflow control nodes would be deployed onto a single EKS node

[nikki-t]

@LucaCinquini - I don't quite understand how to set the instance_class for the aws_db_instance using the Helm charts. I understand the different is for scaling but I don't see a location in the Helm chart that would allow me to set the instance_class. Can you elaborate on what you mean in the first bullet above?

[LucaCinquini]

@nikki-t : if the Helm chart does not support configuring the EC2 instance type (I have not seen such a parameter), why don't we just add a new Terraform variable from the top-level tfvars file, which by default is:

db_instance_class = "db.m5d.xlarge" ?

This way users could overwrite it to something like:

instance_class = "db.m5d.2xlarge"

for a more scalable deployment.