[Maya-Kid]

A spreadsheet stores a date as a number: the days since 1899-12-30, with the time of day as the fraction. It is a date only because the cell's number format says so. calamine reads that format and hands such a cell over as Data::DateTime, but parsers::spreadsheet writes every cell with to_string(), and ExcelDateTime's Display prints the stored number. So every date in an .xlsx or .xls file reaches the base as its day count.

Problem

A workbook written by openpyxl, read on dev with cargo run -p utopia-ingest --example render_doc:

# Sheet: 回款台账

| 合同 | 签订日期 | 到期日 | 回款时间 | 金额（万元） |
| --- | --- | --- | --- | --- |
| 华东-001 | 45306 | 45657 | 45352.39583333334 | 120 |
| 华南-007 | 45351 | 45716 | 45400.58680555555 | 86.5 |

# Sheet: 值班

| 班次 | 开始 | 累计停机 |
| --- | --- | --- |
| 早班 | 0.3333333333333333 | 1.510416666666667 |
| 夜班 | 0.8541666666666666 | 0.03125 |

In the workbook those cells show 2024-01-15, 2024年12月31日, 2024-03-01 9:30:00, 8:00:00 and 36:15:00.

• The date is gone everywhere the text goes: the chunks, evidence quotes, the document page and the full-text index all hold the number.
• Time extraction (0064) cannot read a date that is not in the text. A row like "华东-001 · 签订日期 45306" gives the model a quantity, not a moment.
• The same day is a different number in a workbook that counts from 1904, the old default of Excel for Mac: 43844.

.ods files are not affected, because calamine returns their dates as ISO strings.

Change

parsers::spreadsheet writes a Data::DateTime cell with a new excel_date:

• a whole day as 2024-01-15;
• a date with a time of day as 2024-03-01 09:30, with seconds only when there are some;
• a value whose whole part is 0 as the time of day alone, 08:00, which is what an h:mm format shows;
• a duration, which calamine marks for [h]:mm:ss formats, as elapsed time, 36:15:00;
• a number outside Excel's calendar as before.

Details:

• It uses calamine's own to_ymd_hms_milli. That handles the 1904 epoch and Excel's 1900 leap day. No dependency or feature changes.
• An ISO date is still a figure to table::is_numeric. So rows are classified exactly as before, and only the cell text changes.
• docs/design/sources.md says it in the tables paragraph.
• Documents already imported keep their old text until they are reprocessed.

Verification

Run on macOS (arm64): Rust 1.98.1; the workspace suite against PostgreSQL 16 with pgvector 0.8.2, with Poppler 26.06.

• crates/utopia-ingest/tests/spreadsheet_dates.rs builds an .xlsx the way Excel writes one. Dates are stored as serials under styles.xml formats: 14 (short date), a custom yyyy"年"m"月"d"日", 22 (date and time), 20 (time), 46 (elapsed time) and 4 (a plain number format).
  • a_date_cell_reads_as_the_date_it_shows: | Short date | 2024-01-15 | 1200 | and | Chinese date | 2024-01-16 | 800 |. A General-format 45306 stays 45306.
  • a_time_of_day_and_an_elapsed_time_read_as_clock_time: 2024-01-15 09:30, 09:30, 36:15:00.
  • a_workbook_counting_from_1904_reads_the_same_date: 43844 in a date1904 workbook reads 2024-01-15.
  • On dev with only this file added, all three fail. The cells read 45306, 45307, 45306.395833333336, 0.3958333333333333, 1.5104166666666667 and 43844.
• The openpyxl workbook above now reads as follows:
  • 签订日期 and 到期日: 2024-01-15, 2024-12-31, 2024-02-29 and 2025-02-28;
  • 回款时间: 2024-03-01 09:30 and 2024-04-18 14:05;
  • 开始: 08:00 and 20:30;
  • 累计停机: 36:15:00 and 0:45:00.
• cargo test --locked -p utopia-ingest: 96 unit tests and every integration test pass.
• fmt, clippy and the workspace suite:
  • cargo fmt --all --check and cargo clippy --locked --workspace --all-targets -- -D warnings are clean.
  • UTOPIA_TEST_REQUIRE_DB=1 UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspace passes: 1,331 passed, 0 failed, 6 ignored.
  • That workspace run had this change merged with the three other utopia-ingest fixes opened alongside it: slide line breaks, Word text boxes and Word headings. Nothing outside utopia-ingest reads parsed Office text.

Limits

• calamine does not expose the display format. So a cell formatted to show only a month (yyyy-mm) is written to the day: 2024-01-01.
• calamine does not treat Excel's East Asian built-in date formats (numFmtId 27–36 and 50–58) as dates. A cell stored with one of them still comes through as a number. Custom formats such as yyyy"年"m"月"d"日" are read.

[WaylandYang]

Approving. Thanks @Maya-Kid. Reviewed at 6a0e99f: excel_date leans on calamine's own calendar (1904 epoch and the 1900 leap day included), writes a whole day as a date, a fraction as a time of day, a duration as elapsed time, and leaves anything outside Excel's calendar alone; rows classify as before since an ISO date is still a figure to table::is_numeric. The fixture is built the way Excel writes it, with the Chinese custom format among the styles.