All tests fail on Linux

[jmreicha]

I installed helm, kubectl, and bats on an Ubuntu system and ran the unit test suite bats ./test/unit and all tests failed.

Failed tests

 ✗ client/ConfigMap: enabled by default
   (in test file test/unit/client-configmap.bats, line 11)
     `[ "${actual}" = "true" ]' failed
 ✗ client/ConfigMap: enable with global.enabled false
   (in test file test/unit/client-configmap.bats, line 22)
     `[ "${actual}" = "true" ]' failed
 ✗ client/ConfigMap: disable with client.enabled
   (in test file test/unit/client-configmap.bats, line 32)
     `[ "${actual}" = "false" ]' failed
 ✗ client/ConfigMap: disable with global.enabled
   (in test file test/unit/client-configmap.bats, line 42)
     `[ "${actual}" = "false" ]' failed
 ✗ client/ConfigMap: extraConfig is set
   (in test file test/unit/client-configmap.bats, line 52)
     `[ ! -z "${actual}" ]' failed
 ✗ connectInject/Deployment: disabled by default
   (in test file test/unit/connect-inject-deployment.bats, line 11)
     `[ "${actual}" = "false" ]' failed
 ✗ connectInject/Deployment: enable with global.enabled false
   (in test file test/unit/connect-inject-deployment.bats, line 22)
     `[ "${actual}" = "true" ]' failed
 ✗ connectInject/Deployment: disable with connectInject.enabled
   (in test file test/unit/connect-inject-deployment.bats, line 32)
     `[ "${actual}" = "false" ]' failed
 ✗ connectInject/Deployment: disable with global.enabled
   (in test file test/unit/connect-inject-deployment.bats, line 42)
     `[ "${actual}" = "false" ]' failed
 ✗ connectInject/Deployment: no secretName: no tls-{cert,key}-file set
   (in test file test/unit/connect-inject-deployment.bats, line 52)
     `[ "${actual}" = "false" ]' failed
 ✗ connectInject/Deployment: with secretName: tls-{cert,key}-file set
   (in test file test/unit/connect-inject-deployment.bats, line 77)
     `[ "${actual}" = "true" ]' failed
 ✗ dns/Service: enabled by default
   (in test file test/unit/dns-service.bats, line 11)
     `[ "${actual}" = "true" ]' failed
   /tmp/bats.153734.src: line 11: yq: command not found
 ✗ dns/Service: enable with global.enabled false
   (in test file test/unit/dns-service.bats, line 22)
     `[ "${actual}" = "true" ]' failed
 ✗ dns/Service: disable with dns.enabled
   (in test file test/unit/dns-service.bats, line 32)
     `[ "${actual}" = "false" ]' failed
 ✗ dns/Service: disable with global.enabled
   (in test file test/unit/dns-service.bats, line 42)
     `[ "${actual}" = "false" ]' failed
 ✗ server/ConfigMap: enabled by default
   (in test file test/unit/server-configmap.bats, line 11)
     `[ "${actual}" = "true" ]' failed
 ✗ server/ConfigMap: enable with global.enabled false
   (in test file test/unit/server-configmap.bats, line 22)
     `[ "${actual}" = "true" ]' failed
 ✗ server/ConfigMap: disable with server.enabled
   (in test file test/unit/server-configmap.bats, line 32)
     `[ "${actual}" = "false" ]' failed
 ✗ server/ConfigMap: disable with global.enabled
   (in test file test/unit/server-configmap.bats, line 42)
     `[ "${actual}" = "false" ]' failed
 ✗ server/ConfigMap: extraConfig is set
   (in test file test/unit/server-configmap.bats, line 52)
     `[ ! -z "${actual}" ]' failed
 ✗ server/DisruptionBudget: enabled by default
   (in test file test/unit/server-disruptionbudget.bats, line 11)
     `[ "${actual}" = "true" ]' failed
 ✗ server/DisruptionBudget: enable with global.enabled false
   (in test file test/unit/server-disruptionbudget.bats, line 22)
     `[ "${actual}" = "true" ]' failed
 ✗ server/DisruptionBudget: disable with server.enabled
   (in test file test/unit/server-disruptionbudget.bats, line 32)
     `[ "${actual}" = "false" ]' failed
 ✗ server/DisruptionBudget: disable with server.disruptionBudget.enabled
   (in test file test/unit/server-disruptionbudget.bats, line 42)
     `[ "${actual}" = "false" ]' failed
 ✗ server/DisruptionBudget: disable with global.enabled
   (in test file test/unit/server-disruptionbudget.bats, line 52)
     `[ "${actual}" = "false" ]' failed
 ✗ server/DisruptionBudget: correct maxUnavailable with n=3
   (in test file test/unit/server-disruptionbudget.bats, line 62)
     `[ "${actual}" = "0" ]' failed
 ✗ server/Service: enabled by default
   (in test file test/unit/server-service.bats, line 11)
     `[ "${actual}" = "true" ]' failed
 ✗ server/Service: enable with global.enabled false
   (in test file test/unit/server-service.bats, line 22)
     `[ "${actual}" = "true" ]' failed
 ✗ server/Service: disable with server.enabled
   (in test file test/unit/server-service.bats, line 32)
     `[ "${actual}" = "false" ]' failed
 ✗ server/Service: disable with global.enabled
   (in test file test/unit/server-service.bats, line 42)
     `[ "${actual}" = "false" ]' failed
 ✗ server/Service: tolerates unready endpoints
   (in test file test/unit/server-service.bats, line 53)
     `[ "${actual}" = "true" ]' failed
 ✗ server/StatefulSet: enabled by default
   (in test file test/unit/server-statefulset.bats, line 11)
     `[ "${actual}" = "true" ]' failed
 ✗ server/StatefulSet: enable with global.enabled false
   (in test file test/unit/server-statefulset.bats, line 22)
     `[ "${actual}" = "true" ]' failed
 ✗ server/StatefulSet: disable with server.enabled
   (in test file test/unit/server-statefulset.bats, line 32)
     `[ "${actual}" = "false" ]' failed
 ✗ server/StatefulSet: disable with global.enabled
   (in test file test/unit/server-statefulset.bats, line 42)
     `[ "${actual}" = "false" ]' failed
   /tmp/bats.155386.src: line 42: yq: command not found
 ✗ server/StatefulSet: image defaults to global.image
   (in test file test/unit/server-statefulset.bats, line 52)
     `[ "${actual}" = "foo" ]' failed
 ✗ server/StatefulSet: image can be overridden with server.image
   (in test file test/unit/server-statefulset.bats, line 63)
     `[ "${actual}" = "bar" ]' failed
 ✗ server/StatefulSet: no updateStrategy when not updating
   (in test file test/unit/server-statefulset.bats, line 75)
     `[ "${actual}" = "null" ]' failed
 ✗ server/StatefulSet: updateStrategy during update
   (in test file test/unit/server-statefulset.bats, line 85)
     `[ "${actual}" = "RollingUpdate" ]' failed
 ✗ server/StatefulSet: adds extra volume
   (in test file test/unit/server-statefulset.bats, line 111)
     `[ "${actual}" = "foo" ]' failed
 ✗ server/StatefulSet: adds extra secret volume
   (in test file test/unit/server-statefulset.bats, line 156)
     `[ "${actual}" = "null" ]' failed
 ✗ server/StatefulSet: adds loadable volume
   (in test file test/unit/server-statefulset.bats, line 197)
     `[ "${actual}" = "1" ]' failed
 ✗ ui/Service: enabled by default
   (in test file test/unit/ui-service.bats, line 11)
     `[ "${actual}" = "true" ]' failed
 ✗ ui/Service: enable with global.enabled false
   (in test file test/unit/ui-service.bats, line 23)
     `[ "${actual}" = "true" ]' failed
 ✗ ui/Service: disable with server.enabled
   (in test file test/unit/ui-service.bats, line 33)
     `[ "${actual}" = "false" ]' failed
 ✗ ui/Service: disable with ui.enabled
   (in test file test/unit/ui-service.bats, line 43)
     `[ "${actual}" = "false" ]' failed
 ✗ ui/Service: disable with ui.service.enabled
   (in test file test/unit/ui-service.bats, line 53)
     `[ "${actual}" = "false" ]' failed
 ✗ ui/Service: disable with global.enabled
   (in test file test/unit/ui-service.bats, line 63)
     `[ "${actual}" = "false" ]' failed
 ✗ ui/Service: disable with global.enabled and server.enabled on
   (in test file test/unit/ui-service.bats, line 74)
     `[ "${actual}" = "false" ]' failed
 ✗ ui/Service: no type by default
   (in test file test/unit/ui-service.bats, line 83)
     `[ "${actual}" = "null" ]' failed
 ✗ ui/Service: specified type
   (in test file test/unit/ui-service.bats, line 93)
     `[ "${actual}" = "LoadBalancer" ]' failed

I see yq being used in the unit test suite but not documented anywhere in the README. After installing yq I reran the tests and they still fail.

[mitchellh]

I'm guessing a dependency is missing somewhere. The easiest way to get the tests to pass is to build and run them in this Docker image: https://github.com/hashicorp/consul-helm/blob/master/test/docker/Test.dockerfile That ensures that all the dependencies are there and its what our CI runs the tests in.

I'll document this in the README.

Otherwise, I welcome you to debug the tests further, its almost certainly just a missing or wrong-version dependency. Would be happy to document that but can't do much to debug from here given the output! :)

[jmreicha]

Thanks for the info, I'll give the test Dockerfile a shot and see if I can debug dependencies.

I will report back and open a PR when I get it figured out.

[jmreicha]

@mitchellh Yep totally a dependency issue. There are apparently two different yqs. One is Python based and installed with pip, the other Go based and is installed with apt-get, I was just using the wrong one it looks like.