Skip to content

Commit

Permalink
🔨 [dataset] Handle corrupted cached file in PersistentDataset (#7244)
Browse files Browse the repository at this point in the history
Fixes #5723

### Description
Corrupted cached files in the PersistentDataset cause the exception:
`RuntimeError: Invalid magic number; corrupt file?`

With this PR we handle that case in the try-except block and continue
the usual functionality if the cached file was absent.

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [x] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [x] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: Ishan Dutta <ishandutta0098@gmail.com>
Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
  • Loading branch information
ishandutta0098 and KumoLiu authored Nov 20, 2023
1 parent 882482b commit 24cafad
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions monai/data/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -387,6 +387,12 @@ def _cachecheck(self, item_transformed):
except PermissionError as e:
if sys.platform != "win32":
raise e
except RuntimeError as e:
if "Invalid magic number; corrupt file" in str(e):
warnings.warn(f"Corrupt cache file detected: {hashfile}. Deleting and recomputing.")
hashfile.unlink()
else:
raise e

_item_transformed = self._pre_transform(deepcopy(item_transformed)) # keep the original hashed
if hashfile is None:
Expand Down

0 comments on commit 24cafad

Please sign in to comment.