Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement scale_factor/add_offset CF conformance test, add and align tests #7771

Closed
wants to merge 8 commits into from

Conversation

kmuehlbauer
Copy link
Contributor

  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

This is adding conformance tests for scale_factor/add_offset/dtype when encoding/decoding cf packed data with

class CFMaskCoder(VariableCoder):
. This doesn't include tests for _FillValue/missing_value/valid_range etc. which need a separate test.

The idea behind is two-fold:

  • be strict when encoding to only write CF conforming data, raise Errors if non-conforming
  • be liberal when decoding, but issue warnings

This is another step to move forward to finally fix issues like #2304 and a bunch more.

@kmuehlbauer
Copy link
Contributor Author

OK it seems this is ready for a first round of reviews.

A bit of added context:

Currently there is no dedicated function for checking for CF standard conformance. The idea is to read as much as possible also non-standard conforming data files, but restrict writing non-standard conforming files.

The implemented function ensure_scale_offset_conformance takes a strict keyword argument, which is True when encoding and False when decoding. If strict=True it will raise errors if there is a mismatch with the standard and when strict=False it will issue warnings.

I've only had to adapt a few tests which where not conforming to standard on encoding to align with that. I've observed some warnings in the test suite which we might to have a look into.

One idea would be to fix erroneous scale_factor/add_offset with our best fitting estimate. This is already done for list-type scale_factor/add_offset.

I will follow-up with checks for CFMaskCoder.

@kmuehlbauer
Copy link
Contributor Author

Setting status back to draft for now, still evaluating solutions for the CF encoding/decoding.

@kmuehlbauer kmuehlbauer marked this pull request as draft May 10, 2023 06:23
@kmuehlbauer
Copy link
Contributor Author

I'll close this one. Might resurrect some of the ideas here down the road.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants