Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add e2e runner (go variant) #95

Merged
merged 29 commits into from
Jul 3, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
6e16875
add e2e runner v2
OlegLoewen Jun 14, 2019
7478197
Add support for testcase groups. Merge test results of junit.xml file…
OlegLoewen Jun 24, 2019
e8e0134
Remove unneccessary env variables in test-defs and adjust readme.md
OlegLoewen Jun 25, 2019
c65ad62
Remove unsed functions
OlegLoewen Jun 25, 2019
f1d5a7c
Fix: test summary wrongly calculated if e2e.log file size is big
OlegLoewen Jun 25, 2019
d1a1233
Fix testgrid metadata format
OlegLoewen Jun 26, 2019
15ddde6
Introduce command line flags: kubeconfig, k8sVersion, cloudprovider, …
OlegLoewen Jun 26, 2019
0f5b4df
Move root/test/e2etest to root/integration-tests/e2e
OlegLoewen Jun 28, 2019
305a3f6
Add working.json files to k8s version 1.10, 1.11, 1.12, 1.13
OlegLoewen Jun 28, 2019
f38f8f3
Fix description file generation for untracked testcases
OlegLoewen Jul 1, 2019
cbadeda
Use different archive package which support shifting and overwriting
OlegLoewen Jul 1, 2019
66cdd7c
Fix desc generator calculating untracked testcases
OlegLoewen Jul 1, 2019
e58a9df
Add k8s 1.15.0 description files. Remove not existing testcases.
OlegLoewen Jul 1, 2019
7c8f463
Add newly identified skipped testcases to corresponding skip description
OlegLoewen Jul 1, 2019
124e9fa
Fix e2e test summary calculation
OlegLoewen Jul 1, 2019
72bf452
Trim cmd log to 300 characters
OlegLoewen Jul 1, 2019
07ff2c9
Fix summary failed testcase names array in file
OlegLoewen Jul 1, 2019
2b4e2fa
Get corresponding kubectl
OlegLoewen Jul 1, 2019
6ad549d
Update documentation
OlegLoewen Jul 1, 2019
c0a4969
Fix test defs
OlegLoewen Jul 1, 2019
c7ab20a
Fix path to e2e tests in test-defs
OlegLoewen Jul 1, 2019
69bb5f7
Use golang images isntead of base-step images
OlegLoewen Jul 2, 2019
8a99a65
Move downloaded kubectl statically to /usr/local/bin/kubectl, since k…
OlegLoewen Jul 2, 2019
9d87051
Use customized moveFile function because of invalid cross-device link…
OlegLoewen Jul 2, 2019
22db8cc
add debug arguments
OlegLoewen Jul 2, 2019
98cef82
Improve kubectl installation
OlegLoewen Jul 2, 2019
b441f8c
Remove debug flags
OlegLoewen Jul 2, 2019
af86976
Remove depricated k8s-e2e 1.15 files
OlegLoewen Jul 3, 2019
593df1c
move networktest to integration-tests folder
OlegLoewen Jul 3, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix description file generation for untracked testcases
Fix refactring issues
  • Loading branch information
OlegLoewen committed Jul 3, 2019
commit f38f8f36c679823835c38b2d272740b9938bc7b4
30 changes: 30 additions & 0 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

339 changes: 0 additions & 339 deletions integration-tests/e2e/kubetest/description/1.10/e2e_base_fast.json

This file was deleted.

148 changes: 0 additions & 148 deletions integration-tests/e2e/kubetest/description/1.10/e2e_base_slow.json

This file was deleted.

349 changes: 0 additions & 349 deletions integration-tests/e2e/kubetest/description/1.11/e2e_base_fast.json

This file was deleted.

148 changes: 0 additions & 148 deletions integration-tests/e2e/kubetest/description/1.11/e2e_base_slow.json

This file was deleted.

349 changes: 0 additions & 349 deletions integration-tests/e2e/kubetest/description/1.12/e2e_base_fast.json

This file was deleted.

148 changes: 0 additions & 148 deletions integration-tests/e2e/kubetest/description/1.12/e2e_base_slow.json

This file was deleted.

349 changes: 0 additions & 349 deletions integration-tests/e2e/kubetest/description/1.13/e2e_base_fast.json

This file was deleted.

148 changes: 0 additions & 148 deletions integration-tests/e2e/kubetest/description/1.13/e2e_base_slow.json

This file was deleted.

2 changes: 1 addition & 1 deletion integration-tests/e2e/kubetest/publisher.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ func createMetadataFiles(startedJsonPath, finishedJsonPath string, testSummary S
}

testStatus := "FAILURE"
if TestsuiteSuccessful {
if testSummary.TestsuiteSuccessful {
testStatus = "SUCCESS"
}
finishedJsonContent := []byte(fmt.Sprintf("{\"timestamp\": %d, \"result\": \"%s\", \"metadata\": {\"shoot-k8s-release\": \"%s\", \"gardener\": \"%s\"}}", testSummary.FinishedTime.Unix(), testStatus, config.K8sRelease, config.GardenerVersion))
Expand Down
22 changes: 11 additions & 11 deletions integration-tests/e2e/kubetest/results_evaluator.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,20 +61,20 @@ func analyzeJunitXMLs(junitXMLFilePaths []string, durationSec int, summaryFailed
if err != nil {
return errors.Wrapf(err, "Couldn't unmarshal %s", file.Name())
}
FailedTests += FailedTests
ExecutedTests += ExecutedTests
SuccessfulTests += SuccessfulTests
for _, testcase := range Testcases {
if Skipped {
if _, ok := testcaseNameToTestcase[Name]; !ok {
testcaseNameToTestcase[Name] = testcase
mergedJunitXmlResult.FailedTests += junitXml.FailedTests
mergedJunitXmlResult.ExecutedTests += junitXml.ExecutedTests
mergedJunitXmlResult.SuccessfulTests += junitXml.SuccessfulTests
for _, testcase := range junitXml.Testcases {
if testcase.Skipped {
if _, ok := testcaseNameToTestcase[testcase.Name]; !ok {
testcaseNameToTestcase[testcase.Name] = testcase
}
continue
}
if !Successful {
*summaryFailedTestcases = append(*summaryFailedTestcases, Name)
if !testcase.Successful {
*summaryFailedTestcases = append(*summaryFailedTestcases, testcase.Name)
}
testcaseNameToTestcase[Name] = testcase
testcaseNameToTestcase[testcase.Name] = testcase
testcaseJSON, err := json.MarshalIndent(testcase, "", " ")
if err != nil {
return errors.Wrapf(err, "Couldn't marshal testsuite summary %s", testcaseJSON)
Expand All @@ -88,7 +88,7 @@ func analyzeJunitXMLs(junitXMLFilePaths []string, durationSec int, summaryFailed
}
}
for _, testcase := range testcaseNameToTestcase {
Testcases = append(Testcases, testcase)
mergedJunitXmlResult.Testcases = append(mergedJunitXmlResult.Testcases, testcase)
}
if err := saveJunitXmlToFile(mergedJunitXmlResult); err != nil {
return err
Expand Down