Skip to content

Failing tests in CI, but for some builds still come back as green #319

@spencerahill

Description

@spencerahill

I recently tried running our test suite on a freshly created conda environment for aospy development, and I got some weird errors. To see if they were just a local problem, I re-triggered the Travis builds on our master branch, and they appeared there too. There are two different ones:

  1. On our docs env build, which causes the build to register as a failure.
  2. On all the other builds (e.g. py37), which despite the errors that prevent the test suite from even loading, the build registers as a success. So I suspect that because the tests never even get ran, they can't generate a failure, even though obviously this should be a failure.

So there are three things to fix:

  1. Fix the bug in the docs branch. The error involves an import within dask, and unlike our other builds with dask=1.15.2, the dask version in the docs build is 0.18.2.
  2. Fix the bug in the other branches. I'm pretty sure this stems from enable loading remote hdf5 files pydata/xarray#2782 which made it into xarray v0.12, as when I revert xarray to 0.11, the tests do load on my local machine.
  3. Fix our CI build process so that a failure to load the tests constitutes a failing build.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions