-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
stdatamodels treatment of units #240
Comments
I opened a test PR with a modified This works by:
A test was added to:
This strategy is only necessary for dynamic units (or units where we expect the user might change the unit outside the pipeline). For static units, defining them in the schema is much simpler. Aside from the changes in #243 the test PR has only test and schema changes (no code changes appear to be necessary to make this strategy work however it might be nice to investigate how to avoid needing to define the unit attribute after the table to allow the schemas to be a bit more flexible). |
XREF: spacetelescope/jwst#2869 |
FITS BinTable extensions support using
TUNIT
keywords to define units for columns within the table. The use of these keywords can be abstracted using interfaces like FITS_rec which provide access to these units via thecolumns
attribute (a ColDefs instance).Units are used in jwst code and what follows are a few examples (and by no means an exhaustive list):
FITS_rec
interface to assign units to columns in a table.TUNIT
headers directlyTUNIT
headers for thePATHLOSS
BinTableTUNIT
)There are some considerations when examining how the pipeline uses units:
compatibility with fits and asdf formats
As datamodels should be saveable in both fits and ASDF formats the use of
TUNIT
for saving a unit has some issues.TUNIT
fits_keyword
definition in the schema is ignored when writing an ASDF fileFITS_rec
instances will be converted to structured arrays prior to writing to an ASDF file (losing any units defined in the columns)The last option above (the niswfss_apcorr example) should work for both ASDF and FITS files (in the context of the jwst pipeline). However, opening the table directly in
astropy
(or some other FITS supporting program) will fail to associate units with the table columns as they are not using the standardTUNIT
keyword(s) (in this case usingSIZEUNIT
).attribute interface and model state
Depending on the state of the attribute that contains the table,
stdatamodels
doesn't appear to provide a consistent interface to the table or units. Some initial testing shows:DataModel
will initialize the table using the datatype defined in the schema. This returns anp.ndarray
instance with a structured datatypeFITS_rec
_cast
and ends up as aFITS_rec
after the cast (although the process currently strips the units due to needing to convert the data endinaness to native/little)The text was updated successfully, but these errors were encountered: