Add error handling for index file loading in SearchIndex

[tobiasdiez]

I got the following error while playing with paper-qa:

File D:\Programming\paper-qa\paperqa\agents\search.py:262, in SearchIndex.index_files(self)
    259         async with await anyio.open_file(file_index_path, "rb") as f:
    260             content = await f.read()
    261             self._index_files = pickle.loads(  # noqa: S301
--> 262                 zlib.decompress(content)
    263             )
    264 return self._index_files

error: Error -5 while decompressing data: incomplete or truncated stream

Since it was a bit hard to locate the offending file, I added logging so that one can see where the error is.

(Not sure why the error occurred in the first place, it disappeared after deleting the index files.)

[Copilot]

Pull Request Overview

This PR adds error handling for decompression and unpickling operations when loading index files in the SearchIndex. The changes ensure that if an error occurs during the loading process, an error message is logged so that the offending file can be identified.

• Wrapped the deserialization of index files with a try/except block.
• Added logging to report errors during index file handling.

[Copilot]

Catching all exceptions may obscure underlying issues; consider catching specific exceptions, such as zlib.error or pickle.UnpicklingError, to handle known error scenarios more accurately.

Suggested change

	except Exception:
	logger.exception(f"Failed to load index file {file_index_path}")
	except (pickle.UnpicklingError, zlib.error) as e:
	logger.exception(f"Failed to load index file {file_index_path}: {e}")

[Copilot]

After logging the exception when loading the index file fails, _index_files is not reset, potentially returning stale data. Consider assigning a safe fallback value (e.g., an empty dict) to _index_files in the exception block.

Suggested change

	logger.exception(f"Failed to load index file {file_index_path}")
	logger.exception(f"Failed to load index file {file_index_path}")
	self._index_files = {}

[jamesbraza]

Thank you for this!