Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prow testing #26

Merged
merged 38 commits into from
Apr 11, 2019
Merged
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
cc564f9
verify-subtree.sh: relax check and ignore old content
pohly Mar 6, 2019
849db0a
Merge pull request #8 from pohly/subtree-check-relax
k8s-ci-robot Mar 6, 2019
94fc1e3
build.make: avoid unit-testing E2E test suite
pohly Mar 15, 2019
fb13c51
verify-shellcheck.sh: import from Kubernetes
pohly Mar 15, 2019
e6db50d
check vendor directory
pohly Mar 27, 2019
34010e7
Merge pull request #10 from pohly/vendor-check
k8s-ci-robot Mar 27, 2019
104a1ac
build.make: avoid unit-testing E2E test suite
pohly Mar 15, 2019
3b6af7b
Merge pull request #12 from pohly/local-e2e-suite
k8s-ci-robot Mar 29, 2019
b2d25d4
verify-shellcheck.sh: make it usable in csi-release-tools
pohly Mar 15, 2019
6c7ba1b
build.make: integrate shellcheck into "make test"
pohly Mar 26, 2019
55212ff
initial Prow test job
pohly Mar 15, 2019
2069a0a
Merge pull request #11 from pohly/verify-shellcheck
k8s-ci-robot Apr 2, 2019
0a0fd49
prow.sh: comment clarification
pohly Apr 2, 2019
429581c
prow.sh: pull Go version from travis.yml
pohly Apr 2, 2019
29545bb
prow.sh: take Go version from Kubernetes source
pohly Apr 3, 2019
741319b
prow.sh: improve building Kubernetes from source
pohly Apr 3, 2019
6602d38
prow.sh: different E2E suite depending on Kubernetes version
pohly Apr 3, 2019
d87eccb
prow.sh: switch back to upstream csi-driver-host-path
pohly Apr 3, 2019
95ae9de
Merge pull request #9 from pohly/prow
k8s-ci-robot Apr 3, 2019
2f3433d
.prow.sh: enable testing with Prow script from csi-release-tools
pohly Apr 8, 2019
79f8893
rpc: fix gofmt error
pohly Apr 8, 2019
f501443
prow.sh: AllAlpha=true for unknown Kubernetes versions
pohly Apr 5, 2019
31dfaf3
prow.sh: fix running of just "alpha" tests
pohly Apr 8, 2019
f3d1d2d
prow.sh: fix hostpath driver version check
pohly Apr 8, 2019
aa45a1c
prow.sh: more efficient execution of individual tests
pohly Apr 8, 2019
9b0d9cd
build.make: skip shellcheck if Docker is not available
pohly Apr 8, 2019
546d550
prow.sh: debug failing KinD cluster creation
pohly Apr 8, 2019
cda2fc5
prow.sh: avoid AllAlpha=true
pohly Apr 8, 2019
6617773
Merge pull request #13 from pohly/prow
k8s-ci-robot Apr 9, 2019
7aaac22
prow.sh: remove AllAlpha=all, part II
pohly Apr 10, 2019
c60f382
Merge pull request #14 from pohly/prow
k8s-ci-robot Apr 10, 2019
f14bbd6
Merge commit 'c60f3823c09a501b1cc12b97b20eb05cec9f822e' into prow
pohly Apr 11, 2019
ff9bce4
Replace 'return' to 'exit' to fix shellcheck error
pengzhisun Apr 11, 2019
0c2677e
Merge pull request #15 from pengzhisun/master
k8s-ci-robot Apr 11, 2019
16ca5ce
Merge commit '0c2677e8fecad9bf904fca32b0319068d3eaa186' into prow
pohly Apr 11, 2019
0b10f6a
prow.sh: update csi-driver-host-path
pohly Apr 11, 2019
bcac1c1
Merge pull request #16 from pohly/prow
k8s-ci-robot Apr 11, 2019
881e737
Merge commit 'bcac1c1fb6d791fe54a84716a5fdcccd0987a351' into prow
pohly Apr 11, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
prow.sh: different E2E suite depending on Kubernetes version
Instead of always using the latest E2E tests for all Kubernetes
versions, the plan now is to use the tests that match the Kubernetes
version. However, for 1.13 we have to make an exception because the
suite for that version did not support the necessary
--storage.testdriver parameter.
  • Loading branch information
pohly committed Apr 3, 2019
commit 6602d38bfbce6a526f9225c39aa6b86adbd86978
72 changes: 36 additions & 36 deletions prow.sh
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,19 @@ configvar () {
eval echo "\$3:" "$1=\${$1}"
}

# Prints the value of a variable + version suffix, falling back to variable + "LATEST".
get_versioned_variable () {
local var="$1"
local version="$2"
local value

eval value="\${${var}_${version}}"
if ! [ "$value" ]; then
eval value="\${${var}_LATEST}"
fi
echo "$value"
}

# Go versions can be specified seperately for different tasks
# If the pre-installed Go is missing or a different
# version, the required version here will get installed
Expand Down Expand Up @@ -158,9 +171,15 @@ configvar CSI_PROW_HOSTPATH_CANARY "" "hostpath image"
# all generated files are present.
#
# CSI_PROW_E2E_REPO=none disables E2E testing.
configvar CSI_PROW_E2E_VERSION v1.14.0 "E2E version"
configvar CSI_PROW_E2E_REPO https://github.com/kubernetes/kubernetes "E2E repo"
configvar CSI_PROW_E2E_IMPORT_PATH k8s.io/kubernetes "E2E package"
configvar CSI_PROW_E2E_VERSION_1_13 v1.14.0 "E2E version for Kubernetes 1.13.x" # we can't use the one from 1.13.x because it didn't have --storage.testdriver
configvar CSI_PROW_E2E_VERSION_1_14 v1.14.0 "E2E version for Kubernetes 1.14.x"
# TODO: add new CSI_PROW_E2E_VERSION entry for future Kubernetes releases
configvar CSI_PROW_E2E_VERSION_LATEST master "E2E version for Kubernetes master" # testing against Kubernetes master is already tracking a moving target, so we might as well use a moving E2E version
configvar CSI_PROW_E2E_REPO_LATEST https://github.com/kubernetes/kubernetes "E2E repo for Kubernetes >= 1.13.x" # currently the same for all versions
configvar CSI_PROW_E2E_IMPORT_PATH_LATEST k8s.io/kubernetes "E2E package for Kubernetes >= 1.13.x" # currently the same for all versions
configvar CSI_PROW_E2E_VERSION "$(get_versioned_variable CSI_PROW_E2E_VERSION "${csi_prow_kubernetes_version_suffix}")" "E2E version"
configvar CSI_PROW_E2E_REPO "$(get_versioned_variable CSI_PROW_E2E_REPO "${csi_prow_kubernetes_version_suffix}")" "E2E repo"
configvar CSI_PROW_E2E_IMPORT_PATH "$(get_versioned_variable CSI_PROW_E2E_IMPORT_PATH "${csi_prow_kubernetes_version_suffix}")" "E2E package"

# csi-sanity testing from the csi-test repo can be run against the installed
# CSI driver. For this to work, deploying the driver must expose the Unix domain
Expand Down Expand Up @@ -200,42 +219,22 @@ regex_join () {
echo "$@" | sed -e 's/ */|/g' -e 's/^|*//' -e 's/|*$//' -e 's/^$/this-matches-nothing/g'
}

# Prints the value of a variable + version suffix, falling back to variable + "LATEST".
get_versioned_variable () {
local var="$1"
local version="$2"
local value

eval value="\${${var}_${version}}"
if ! [ "$value" ]; then
eval value="\${${var}_LATEST}"
fi
echo "$value"
}

# Which tests are alpha depends on the Kubernetes version. The same
# E2E test suite is used for all Kubernetes versions, including older
# Kubernetes.
#
# Feature tags in the test are set based on what is an alpha
# feature in the Kubernetes version that contains the E2E test suite's
# source code.
# Which tests are alpha depends on the Kubernetes version. We could
# use the same E2E test for all Kubernetes version. This would have
# the advantage that new tests can be applied to older versions
# without having to backport tests.
#
# So when testing against an older Kubernetes release, some tests
# might be enabled by default which don't pass for that older
# Kubernetes version. In that case, the
# CSI_PROW_E2E_ALPHA_<k8s-version> variable cannot just
# be based on the `Feature` tag, it also must contain the names
# of tests that used to be alpha but no longer have that tag.
# But the feature tag gets removed from E2E tests when the corresponding
# feature becomes beta, so we would have to track which tests were
# alpha in previous Kubernetes releases. This was considered too
# error prone. Therefore we use E2E tests that match the Kubernetes
# version that is getting tested.
#
# <k8s-version> is just major+minor version separated by
# underscore. "latest" matches master. It is also used for
# unknown versions, so when master gets released there is
# no need to add a new versioned variable. That only needs to be
# done when updating something in the configuration that leads
# to "latest" no longer being suitable.
# However, for 1.13.x testing we have to use the E2E tests from 1.14
# because 1.13 didn't have --storage.testdriver yet, so for that (and only
# that version) we have to define alpha tests differently.
configvar CSI_PROW_E2E_ALPHA_1_13 '\[Feature: \[Testpattern:.Dynamic.PV..block.volmode.\] should.create.and.delete.block.persistent.volumes' "alpha tests for Kubernetes 1.13" # Raw block was an alpha feature in 1.13.
configvar CSI_PROW_E2E_ALPHA_LATEST '\[Feature:' "alpha tests for Kubernetes master"
configvar CSI_PROW_E2E_ALPHA_LATEST '\[Feature:' "alpha tests for Kubernetes >= 1.14" # there's no need to update this, adding a new case for CSI_PROW_E2E for a new Kubernetes is enough
configvar CSI_PROW_E2E_ALPHA "$(get_versioned_variable CSI_PROW_E2E_ALPHA "${csi_prow_kubernetes_version_suffix}")" "alpha tests"

# After the parallel E2E test without alpha features, a test cluster
Expand All @@ -251,6 +250,7 @@ configvar CSI_PROW_E2E_ALPHA "$(get_versioned_variable CSI_PROW_E2E_ALPHA "${csi
# the failing test for "latest" or by updating the test and not running
# it anymore for older releases.
configvar CSI_PROW_E2E_ALPHA_GATES_1_13 'VolumeSnapshotDataSource=true,BlockVolume=true,CSIBlockVolume=true' "alpha feature gates for Kubernetes 1.13"
# TODO: add new CSI_PROW_ALPHA_GATES entry for future Kubernetes releases
configvar CSI_PROW_E2E_ALPHA_GATES_LATEST 'VolumeSnapshotDataSource=true' "alpha feature gates for latest Kubernetes"
configvar CSI_PROW_E2E_ALPHA_GATES "$(get_versioned_variable CSI_PROW_E2E_ALPHA_GATES "${csi_prow_kubernetes_version_suffix}")" "alpha E2E feature gates"

Expand Down