Skip to content

Load concatenated split CSV caches when primary file is missing and add tests#64

Merged
hemanth-asirvatham merged 1 commit intomainfrom
find-timeout-retries-for-get-all-responses
Feb 25, 2026
Merged

Load concatenated split CSV caches when primary file is missing and add tests#64
hemanth-asirvatham merged 1 commit intomainfrom
find-timeout-retries-for-get-all-responses

Conversation

@hemanth-asirvatham
Copy link
Collaborator

Motivation

  • Allow the API to recover cached outputs when a single final CSV is not present but split parts like name_1.csv, name_2.csv, ... exist.
  • Ensure deterministic ordering when reassembling split files so downstream consumers get a consistent DataFrame.
  • Provide unit tests that cover both the split-file fallback and the preference for a primary file when it exists.

Description

  • Added Path import and a helper _find_split_parts that discovers and numerically sorts split files matching the pattern stem_<index><suffix> in the same directory as the target path.
  • Extended _load_cached_dataframe to detect and load split CSV parts, concat them with pd.concat(..., ignore_index=True), and print a diagnostic message when using split results.
  • Preserved the existing behavior of preferring the primary final_path file when present and raising FileNotFoundError only if neither the primary nor split parts are available.
  • Added tests/test_api_cached_loading.py with two tests that validate the split-file fallback ordering and the primary-file preference.

Testing

  • Ran the new unit tests in tests/test_api_cached_loading.py using pytest, and both tests passed.
  • Existing code paths that read a single CSV remain covered by the new tests which confirm the primary file is preferred when present.

Codex Task

@hemanth-asirvatham hemanth-asirvatham merged commit 81e9934 into main Feb 25, 2026
1 check failed
@github-actions
Copy link


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@github-actions github-actions bot locked and limited conversation to collaborators Feb 25, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant