Terraform is used to provision scenarios for integration testing of some cloud features. Features implementing integration tests that require the presence of cloud resources should have their own Terraform configuration, this configuration can be used when developing locally to create (and destroy) resources that allow to test these features.
Tests requiring access to cloud providers should be disabled by default with the use of build tags.
Terraform is available in https://www.terraform.io/downloads.html
Download it and place it in some directory in your PATH.
terraform is the main command for Terraform and the only one that is usually
needed to manage configurations. Terraform will also download other plugins that
implement the specific functionality for each provider. These plugins are
automatically managed and stored in the working copy, if you want to share the
plugins between multiple working copies you can manually install them in the
user the user plugins directory located at ~/.terraform.d/plugins,
or %APPDATA%\terraform.d\plugins on Windows.
Plugins are available in https://registry.terraform.io/
The most important commands when using Terraform are:
* terraform init to do some initial checks and install the required plugins.
* terraform apply to create the resources defined in the configuration.
* terraform destroy to destroy resources previously created.
Cloud providers use to require credentials, they can be provided with the usual methods supported by these providers, using environment variables and/or credential files.
Terraform stores the last known state of the resources managed by a
configuration in a terraform.tfstate file. It is important to keep this file
as it is used as input by terraform destroy. This file is created in the same
directory where terraform apply is executed.
Please take a look to Terraform documentation for more details: https://www.terraform.io/intro/index.html
The main purpouse of Terraform in Beats is to create and destroy cloud resources
required by integration tests. For these configurations there are some things to
take into account:
* Apply should work without additional inputs or files. Only input will be the
required for specific providers, using environment variables or credential
files.
* You must be able to apply the same configuration multiple times in the same
account. This will allow to have multiple builds using the same configuration
but with different instances of the resources. Some resources are already
created with unique identifiers (as EC2 instances), some others have to be
explicitly created with unique names (e.g. S3 buckets). For these cases random
suffixes can be added to identifiers.
* Destroy must work without additional input, and should be able to destroy all
the resources created by the configuration. There are some resources that need
specific flags to be destroyed by terraform destroy. For example S3 buckets
need a flag to force to empty the bucket before deleting it, or RDS instances
need a flag to disable snapshots on deletion.
Integration tests that need the presence of certain resources to work can be executed in CI if they provide a Terraform configuration to start these resources. These tests are disabled by default in CI.
Terraform states are archived as artifacrs of builds, this allows to manually destroy resources created by builds that were not able to do a proper cleanup.
Here is a checklist to add support for a cloud feature in Jenkins:
* In the feature code:
* Tests have a build tag so they are disabled by default. When run from mage,
its execution can be selected using the TEST_TAGS environment variable, e.g:
TEST_TAGS=aws for AWS tests.
* There is some Terraform configuration that defines a cloud scenario where
tests pass. This configuration should be in the directory of the feature.
* In the Jenkinsfile:
* Add a boolean parameter to run the tests on this environment, e.g.
awsCloudTests. This parameter should be set to false by default.
* Add a conditional block in withCloudTestEnv that:
* Will be executed if the previously added boolean parameter, or allCloudTests
are set to true.
* Adds the tag to TEST_TAGS (as comma separated values), so tests are
selected.
* Defines how to obtain the credentials and provide them to the tests.
* In the stage of the specific beat:
* Add a stage that calls to startCloudTestEnv, if there isn’t anyone.
* Add a post cleanup step that calls to terraformCleanup, if there isn’t anyone.
* Add a environment to the list of environments started by startCloudEnv,
with the condition to start the scenario, and the path to the directory
with its definition, e.g. [cond: params.awsCloudTests, dir: 'x-pack/metricbeat/module/aws']