Integration tests for testmachinery controller

Overview Execution procedure

• make install: install prerequisites of the tm controller which includes argo, minio and the testrun crd
• make install-controller: install the testmachinery-controller with necessary rbac rules and validation webhook
• run integration tests
• make remove-controller
• make remove

Test validation webhook

• validation failing:

  • location:
    • missing locations
    • local testrun
  • testflow:
    • no testdefs found or defined
    • label with no testdefs
  • invalid kubeconfig syntax (no connection test)

• validating succeed

  • no testruns in onExit

Controller tests

• testflow with name
• testflow with label
• onExit
  • success condition (called when testrun succeeds, not called when testrun fails)
  • failing condition
  • always
• TTL termination after workflow is completed
• TTL should wait until workflow has finished
• garbage collection on delete

Testrunner tests

• Test result summary as elastic search bulk request.
• Test result summary with exported artifacts of specific test steps.