Detect overall PCT failures #277
Conversation
jenkinsci#179 (comment) Unfortunately TEST_FAILURES only asserts that surefire:test did not succeed, but does not verify that it even ran to begin with: https://github.com/jenkinsci/plugin-compat-tester/blob/c502ba1cc6ebf39b9525e914fc537c38918ed535/plugins-compat-tester/src/main/java/org/jenkins/tools/test/PluginCompatTester.java#L322-L323 (cherry picked from commit b489a73)
(cherry picked from commit 91fdadd)
(cherry picked from commit 6b68069)
…ill leave no test files behind; really better for PCT to set -Dmaven.test.failure.ignore
|
My browser just chokes trying to look at the log, and blueocean keeps jumping to the bottom -.- |
|
@jglick LGTM, I can confirm that is an issue that we have solved in other places in similar ways |
|
@jglick I'd like to understand how to get to those partial logs. |
We could |
|
@jglick I was more referring on how to navigate to them in the Jenkins UI |
|
Go to Pipeline Steps, search for the branch corresponding to the test failure, look for the |
|
So, now we have three failures. Looking at https://ci.jenkins.io/job/Tools/job/bom/job/PR-277/3/ here are the logs pulled out:
I'm not even sure how to read the failures. When this kind of failure occurs do we want/need to rerun with |
|
the ssh-credentials one looks like a straightforward one to do with the jcasc test harness |
You can run PCT for one LTS line and plugin locally quite easily: https://github.com/jenkinsci/bom/#pct The |
|
…which is blocked in #274 on some weirdness I have not dealt with yet: #251 (comment) |
|
The |
|
Flakes? |
When PCT compilation succeeded but
surefire:testor one of its preparatory goals crashed (i.e., not just some test failures), we were silently skipping that plugin, and it seems there were several of these. Noticed in #179. At root this seems like a flaw in PCT which I am just working around here.