-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
silx.io.fioh5: Improved handling of missing data entries #4156
base: main
Are you sure you want to change the base?
Conversation
Fix CI failures
Could you also add a unit test with FIO data that has missingvalues? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR!
As already mentioned, the continuous integration fails highlighting an bytes/str issue and it would also be great to have a test with nan
and no data
"""Convert a byte of truth to true (1) or false (0). | ||
True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values | ||
are 'n', 'no', 'f', 'false', 'off', and '0'. Raises ValueError if | ||
'val' is anything else. | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need for a complete description in the docstring IMO, the code is self-explanatory
"""Convert a byte of truth to true (1) or false (0). | |
True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values | |
are 'n', 'no', 'f', 'false', 'off', and '0'. Raises ValueError if | |
'val' is anything else. | |
""" | |
"""Convert bytes of a truth value to bool. | |
Raises ValueError if 'val' is not supported. | |
""" |
elif val in ('n', 'no', 'f', 'false', 'off', '0'): | ||
return False | ||
else: | ||
raise ValueError("invalid truth value %r" % (val,)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
raise ValueError("invalid truth value %r" % (val,)) | |
raise ValueError("Invalid truth value %r" % val) |
Enables the loading of fio files with missing data entries, that can not be converted into the target data type.
Mostly uses the default behaviour of numpy.genfromtxt.
For example: Entries like
<no data>
ornan
in a FLOAT data column are automatically converted tonumpy.nan
bynumpy.genfromtxt
.Not all missing values can be handled:
Missing boolean or integer values cannot be flagged as missing (i.e.
nan
) in a bool or integer array and thus result in aValueError
.Closes #4155