Skip to content

Conversation

@kylebarron
Copy link
Member

@kylebarron kylebarron commented Nov 22, 2025

Replace simple prefetch buffer with exponential readahead cache for more efficient metadata fetching, and improve Python error handling.

Supersedes and closes #83, closes #89

Change list

  • ReadaheadMetadataCache grows fetch sizes (configurable via initial_size and multiplier parameters)
  • Caches sequential blocks from file start to minimize redundant network requests
  • Default configuration: 32 KiB initial fetch with 2x multiplier for subsequent fetches
  • Updates Python bindings
  • Adds minimal Python error handling via PyAsyncTiffError enum, so the existing test that checked for FileNotFoundError would still pass. Better exception handling is tracked in Improved Python error handling #144

@github-actions github-actions bot added the feat label Nov 22, 2025
Copy link
Member

@weiji14 weiji14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend to add a unit test 🙂

Comment on lines +60 to +61
let chunk = b.slice(start..end);
out_buffers.push(chunk);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The slice here might return an empty chunk, and the .push would fail with "IOError(Error { kind: UnexpectedEof, message: "failed to fill whole buffer" })". Maybe re-check the logic here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you think of an example of a unit test that would catch this?

@kylebarron kylebarron changed the title feat: Exponential cache feat: Exponential read-ahead cache Nov 23, 2025
@kylebarron
Copy link
Member Author

I added some unit tests in 00a6e72 (#140)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tokio hangs when opening TIFF

3 participants