Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Netlib BLAS test failures should cause CI failure #518

Open
devinamatthews opened this issue Jul 6, 2021 · 10 comments
Open

Netlib BLAS test failures should cause CI failure #518

devinamatthews opened this issue Jul 6, 2021 · 10 comments
Assignees

Comments

@devinamatthews
Copy link
Member

See e.g. here. The build passes even though many BLAS tests fail. In this case, I have no idea why they fail but since they do so only for shared builds and not static it is probably a real bug that the CI workflow should indicate.

For context, similar BLAS test failures happen on Windows/AVX512 (#514) and are not reflected in the exit code of make check.

@isuruf
Copy link
Contributor

isuruf commented Jul 6, 2021

The way the BLAS tests are setup, it's impossible to have them run on windows with shared builds.

This is because a local xerbla implementation needs to override the one in the blis library which works in Unix and in windows static builds, but with windows DLLs, the xerbla implementation in the DLL takes precedence.

@fgvanzee
Copy link
Member

fgvanzee commented Jul 6, 2021

Ah yes, I forgot about that highly unfortunate feature of the BLAS test drivers. Thanks for that reminder, Isuru.

@devinamatthews
Copy link
Member Author

So is the symptom of this seemingly random test failures or is there a more specific outcome?

@isuruf
Copy link
Contributor

isuruf commented Jul 6, 2021

So is the symptom of this seemingly random test failures

Yes

@devinamatthews
Copy link
Member Author

Well, that sucks. I'll revert the changes to .appveyor.yml but I still think that BLAS tests should trigger CI failure. I've have them fail in other circumstances where the BLIS tests did not and there was indeed a bug.

@isuruf
Copy link
Contributor

isuruf commented Jul 6, 2021

With static libraries, it should work though.

@fgvanzee
Copy link
Member

fgvanzee commented Jul 7, 2021

I recommend that we insert a comment somewhere (in the .appveyor.yml file?) that will remind future readers of this issue.

@devinamatthews
Copy link
Member Author

I recommend that we insert a comment somewhere (in the .appveyor.yml file?) that will remind future readers of this issue.

Done

@devinamatthews
Copy link
Member Author

Apparently all test failures are ignored in the Makefile (prepended with "- "). @fgvanzee can you remind me what the rationale for this is? Makes it hard for other people to incorporate "make check" into their CI pipelines.

@fgvanzee
Copy link
Member

fgvanzee commented Jul 7, 2021

Apparently all test failures are ignored in the Makefile (prepended with "- "). @fgvanzee can you remind me what the rationale for this is?

I probably did this because I wanted make to continue whatever rule is being executed, even if an error is encountered.

Here's the relevant part of the GNU make documentation, for those who are curious.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants