crds.getreferences does not verify local files

[braingram]

If a file exists in the local cache, the file is not verifid on a call to getreferences. Using the example in this issue below, when run the first time (assuming no corresponding file exists) the file will be downloaded and verified as seen here

crds/crds/client/api.py

Lines 673 to 680 in aa0d8f3

	def download_core(self, name, localpath):
	"""Download and verify file `name` under context `pipeline_context` to `localpath`."""
	if config.get_download_plugin():
	self.plugin_download(name, localpath)
	else:
	generator = self.get_data_http(name)
	self.generator_download(generator, localpath)
	self.verify_file(name, localpath)

If run a second time (when the file exists) the file will be opened and not verified. This can be confirmed by replacing the file with a blank file of the same name.

import logging
import crds

logging.basicConfig(level=logging.DEBUG)
logging.root.setLevel(logging.DEBUG)
dd = {
    'roman.meta.instrument.name': 'WFI',
    'roman.meta.instrument.detector': 'WFI02',
     'roman.meta.exposure.start_time': '2021-01-01T00:00:00.000',
}

crds.getreferences(dd, reftypes=('mask',), observatory='roman')

@stscieisenhamer came up with the initial test (replacing the file with a blank file) and mentioned that this issue can result in an incorrect local crds cache if multiple processes attempt to fetch the same file (crds attempts to protect against this but may not be perfect). @zacharyburnett may have fixed the jwst regression tests to improve the reliability of crds syncing and these improvements might be port-able to roman to improve the reliability of regression tests.

[braingram]

This issue came up again while testing with crds-test. The nrm reference file jwst_niriss_nrm_0002.fits was renamed on the server to jwst_niriss_nrm_0001.fits. Since I already had a local cache which contained the old jwst_niriss_nrm_0001.fits crds failed to download the new version.