Skip to content

CodeRabbit: Configure non-collapsed summary #147

Description

@amotl

About

Currently, the auto-generated PR summary is collapsed into the walkthrough. I'd like to configure it like it was before, that it is represented by a full dedicated comment.

Details

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added support for ingesting XLSX files from local filesystems and cloud blob sources.
    • XLSX is now recognized as a supported file format in endpoint selection and format messaging.
    • Excel sheets can now be read and converted into rows for downstream processing.
  • Tests

    • Added coverage for Excel ingestion and updated existing checks to reflect the expanded supported formats.

Walkthrough

This PR adds XLSX file format support to the filesystem connector. It introduces a _read_excel reader using polars, a read_excel standalone transformer, and dispatch logic in resource_for_reader. The table argument is threaded through blobstorage and filesystem source APIs as sheet_name. The xlsx format is registered in FORMAT_TO_READER, new dependencies (fastexcel, polars) are added, and tests plus changelog entries are updated accordingly.

Changes

XLSX Reader Support

Layer / File(s) Summary
Excel reader implementation and adapter dispatch
omniload/source/filesystem/readers.py, omniload/source/filesystem/adapter.py
Adds _read_excel helper reading XLSX via polars, exposes read_excel standalone transformer, and extends resource_for_reader to special-case read_excel by passing table as sheet_name.
Table argument wiring through source APIs
omniload/source/blobstorage/api.py, omniload/source/filesystem/api.py, tests/main/test_file_source.py
GCSSource, S3Source, and LocalFilesystemSource now pass table into resource_for_reader; test stub updated to capture the new argument.
Format registration, dependencies, and validation
omniload/util/endpoint.py, pyproject.toml, tests/main/test_blob.py, tests/warehouse/storage/test_excel.py, docs/changelog.md
Registers xlsxread_excel mapping, adds fastexcel/polars dependencies, adds an ingestion test for Excel, updates supported-format message test, and adds a changelog entry.

Estimated code review effort: 2 (Simple) | ~15 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Source as FilesystemSource
  participant Adapter as resource_for_reader
  participant Reader as _read_excel
  participant Polars as polars.read_excel

  Source->>Adapter: resource_for_reader(reader_name="read_excel", table)
  Adapter->>Reader: read_excel(table as sheet_name)
  Reader->>Polars: read_excel(file bytes, sheet_name)
  Polars-->>Reader: parsed rows
  Reader-->>Adapter: yield TDataItems
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: adding XLSX source adapter support.
Description check ✅ Passed The description is related to the change and accurately notes the missing XLSX spreadsheet support.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch excel

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Originally posted by @coderabbitai[bot] in #123 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationrefinementSmall steps for enhanced robustness and details done right.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions