Add a terraform subcommand

[plkokanov]

What would you like to be added:
We should add scripts that can ease work with terraform states.

Why is this needed:
Sometimes there are errors caused by terraform job pods:

1. A resource has already been created by a previous execution but the terraform state was not properly saved
2. There are resource leaks due to multiple executions of terraform jobs

To fix these the operator has to either manually delete the created resources on the cloud provider or manually update the terraform state configmaps kept in the Shoot's namespace on the Seed.

To ease this work we can introduce a couple of scripts which take care of automatically setting up the terraform work directory and properly updating the state's configmap.

[plkokanov]

Here's what I'm planning for the commands to look like:

# After targeting a shoot cluster
# Initialize terraform working directory. 
# The directory will be initialized under ~/.garden/cache/<garden-cluster>/seeds/<seed-name>/<shoot-name>/terraform
# or ~/.garden/cache/<garden-cluster>/projects/<project-name>/<shoot-name>/terraform
# All relevant terraform configmaps will be downloaded in that directory. Secrets will remain in the cluster
$ gardenctl terraform init

# Import an existing resource to the terraform state.
$ gardenctl terraform import google_compute_network.network <shoot-network-name>

# Push the modified state to the control plane of the shoot. 
# Pushing will show a diff between the local and remote states and a confirmation will be necessary before continuing. 
# Confirmation can be skipped by specifying -f flag.
$ gardenctl terraform push 

@vlerenc @DockToFuture wdyt?

[DockToFuture]

Sounds good to me! Regarding the first part, we have already implemented a download function for the terraform files, see gardenctl download tf infra. Maybe you could reuse the code for that or factor it out.

[ialidzhikov]

Hmm, I am with mixed feelings about it. With g terraform an operator needs to run the exact terraform version (and probably the exact provider version) that is used in the current terraformer (used in the current g/g-ext). Otherwise most likely there will be some incompatibilities such as new version of terraform state, unsupported syntax features and etc.
g aws or any cloud provider cli is frequently used during operations because an operator needs to check for unmanaged resource and etc. An operator would need to perform terraform import in some edge cases and most likely there is an issue that cause the terraform state loss.
Also I assume terraform as an implementation detail- an extension provider could implement the infrastructure with native sdk calls.

Personally I would go with mkdir foo; cd foo, get the main.tf, terraform.state and plan, import, apply (if needed).

[DockToFuture]

The tf version validation is crucial here, if we don't want to break things up. And even with a valid validation such commands should only be used with precaution. But there shouldn't be a difference if you do it in an automated way or manual.
Terraform was never the "sustainable solution" but a good start in the beginning.

[plkokanov]

@ialidzhikov this is one of the reasons I decided to open the issue as a hack/script for the ops-toolbelt after initially considering it for gardenctl.

Generally, running terraform init will report errors if there is a missmatch between the terraform versions used to handle the main.tf files.

The convenient thing you would get from gardenctl when using terraform is easier credential management and automatic terraform working directory creation with a consistent path (depending on gardener, project, seed and shoot clusters) which we already have with gardenctl download tf.