Re-segment fixed-layout content into one sentence per element - #877
Draft
mickael-menu wants to merge 2 commits into
Draft
Re-segment fixed-layout content into one sentence per element#877mickael-menu wants to merge 2 commits into
mickael-menu wants to merge 2 commits into
Conversation
Fixed-layout publications (PDF, EPUB FXL) cut sentences at page breaks, making TTS speak broken fragments interleaved with page numbers and running headers. `DefaultContentService` now wraps the publication iterator in a new `SentenceStitchingContentIterator` for fixed-layout publications. It re-balances text across page seams as linked per-page segments marked `continued`, and marks page artifacts (page numbers, running headers) detected by extensible `PageArtifactDetector`s. A new internal `BufferedContentIterator` absorbs the resource iterators' cursor bookkeeping. `PublicationSpeechSynthesizer` groups cross-page sentences into single utterances with per-part locators (`Utterance.parts`), skips page artifacts, and narrows the spoken range inside the part containing it so the navigator turns the page when speech crosses the seam. The Test App now applies one decoration per part. Also aligns `PDFResourceContentIterator` with the HTML iterator for a progression of 1.0 (start past the last page), so backward iteration across resources no longer skips the last page. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the seam-only sentence stitching iterator with a full
re-segmentation pipeline for fixed-layout publications (PDF, EPUB FXL):
- Raw elements are decomposed into fragments (PDF printed lines, FXL
block segments). Page numbers and running headers become standalone
`pageArtifact` elements, and the new extensible `HardBreakDetector`
keeps display text ("P A R T O N E", "Chapter 1", title pages) out of
sentences.
- Body fragments are joined into normalized logical text (de-hyphenated
word cuts, CJK-aware joins) and re-tokenized so each
`TextContentElement` holds exactly one sentence, with one segment per
fragment locatable on its own page. Sentences merge across printed
lines, sibling FXL blocks and up to 4 pages.
- Emitted elements carry `sentenceAligned` and bypass
`makeTextContentTokenizer`, preserving on-page highlights for
decorations.
- `ContentSearchService` skips page artifacts by default
(`ignoresPageArtifacts`) so queries spanning a page boundary match.
- Forward and backward iteration produce identical streams from any
starting point; all derivations are pure functions of a bounded raw
window.
Docs, ADR 0001, glossary and CHANGELOG updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR improves fixed-layout (PDF, EPUB FXL) reading flow by re-segmenting content into one sentence per TextContentElement, so TTS and search work across printed line breaks, block boundaries, and (bounded) page seams while filtering page-boundary noise.
Changes:
- Added
SentenceContentIterator+SentenceRegionBuilderpipeline to fragment, classify (page artifacts / hard breaks), join, and re-tokenize fixed-layout text into sentence-aligned elements. - Updated
PublicationSpeechSynthesizerto skip page artifacts and support multi-part utterances with per-part locators for cross-page highlighting and page turns. - Updated
ContentSearchServiceto optionally ignore page artifacts (default: true) and documented the fixed-layout normalized/snippet behavior.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| Tests/SharedTests/Publication/Services/Content/PageArtifactDetectorTests.swift | Adds tests for page artifact detectors. |
| Tests/SharedTests/Publication/Services/Content/Iterators/SentenceContentIteratorTests.swift | End-to-end tests for sentence re-segmentation across lines/pages/resources. |
| Tests/SharedTests/Publication/Services/Content/Iterators/PDFResourceContentIteratorTests.swift | Updates tests for new “start past end at progression 1.0” semantics. |
| Tests/SharedTests/Publication/Services/Content/Iterators/BufferedContentIteratorTests.swift | Adds tests for the new buffering/peeking iterator wrapper. |
| Tests/SharedTests/Publication/Services/Content/HardBreakDetectorTests.swift | Adds tests for hard break detection rules. |
| Tests/SharedTests/Publication/Services/Content/DefaultContentServiceTests.swift | Verifies iterator selection (plain vs sentence iterator) by publication type. |
| Tests/SharedTests/Publication/Services/Content/ContentTokenizerTests.swift | Ensures sentence-aligned elements bypass re-tokenization. |
| Tests/NavigatorTests/TTS/PublicationSpeechSynthesizerUtteranceTests.swift | Tests utterance multi-part locator mapping and seam behavior. |
| TestApp/Sources/Reader/Common/TTS/TTSViewModel.swift | Updates highlighting to apply one decoration per utterance part. |
| Sources/Shared/Publication/Services/Search/ContentSearchService.swift | Adds ignoresPageArtifacts option and filters artifacts during windowing/search. |
| Sources/Shared/Publication/Services/Content/PageArtifactDetector.swift | Introduces artifact kinds, continuation joiners, attribute keys, and detectors. |
| Sources/Shared/Publication/Services/Content/Iterators/SentenceRegionBuilder.swift | Implements fragmentation, classification, joining, and sentence output generation. |
| Sources/Shared/Publication/Services/Content/Iterators/SentenceContentIterator.swift | Adds the fixed-layout re-segmentation iterator with windowing/anchoring. |
| Sources/Shared/Publication/Services/Content/Iterators/PDFResourceContentIterator.swift | Adjusts end-start semantics for progression 1.0 for backward iteration consistency. |
| Sources/Shared/Publication/Services/Content/Iterators/BufferedContentIterator.swift | Adds buffering/peeking wrapper honoring cursor invariants. |
| Sources/Shared/Publication/Services/Content/HardBreakDetector.swift | Introduces hard break detectors and terminal punctuation helper. |
| Sources/Shared/Publication/Services/Content/ContentTokenizer.swift | Skips tokenization for .sentenceAligned elements to preserve highlights. |
| Sources/Shared/Publication/Services/Content/ContentService.swift | Applies sentence re-segmentation automatically for fixed-layout/PDF content. |
| Sources/Shared/Publication/Services/Content/Content.swift | Documents cursor semantics for ContentIterator. |
| Sources/Navigator/TTS/PublicationSpeechSynthesizer.swift | Adds utterance parts and uses per-part locators for highlights/page turns; skips artifacts. |
| docs/Guides/TTS.md | Documents fixed-layout utterance parts and highlighting/page-turn behavior. |
| docs/Guides/Search.md | Documents normalized snippets and ignoresPageArtifacts tradeoff. |
| docs/Guides/Content.md | Documents fixed-layout sentence re-segmentation, artifacts, hard breaks, extensibility. |
| docs/adr/0001-sentence-resegmentation-iterator.md | Records design decision/constraints for the iterator-based approach. |
| CONTEXT.md | Adds glossary for key terms (seam/fragment/region/anchor/etc.). |
| CHANGELOG.md | Adds unreleased entries for new fixed-layout sentence re-segmentation, TTS/search changes, and PDF iterator fix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+27
to
+31
| "Page 42", | ||
| "page 42", | ||
| "p. 42", | ||
| "42 / 300", | ||
| "42 of 300", |
Comment on lines
+308
to
+310
| private func isSearchable(_ element: ContentElement) -> Bool { | ||
| !ignoresPageArtifacts || element.attribute(.pageArtifact) == nil | ||
| } |
| public let text: String | ||
| /// Locator to the utterance in the publication. | ||
| public let locator: Locator | ||
| /// Language of this utterance, if it dffers from the default publication language. |
Comment on lines
+134
to
+140
| // "Page 42", "page 42", "p. 42", "P42". | ||
| for prefix in ["page", "p."] { | ||
| if text.lowercased().hasPrefix(prefix) { | ||
| text = String(text.dropFirst(prefix.count)) | ||
| .trimmingCharacters(in: .whitespaces) | ||
| } | ||
| } |
stevenzeck
approved these changes
Aug 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
TTS is unusable with PDF and EPUB FXL publications: sentences are cut by printed line breaks (
NLTokenizersplits at every\nin a PDF page blob), by sibling block elements (one sentence spans several<div>s of an FXL page), and by page boundaries, so the synthesizer speaks broken fragments ("…there is a particu-"), with page numbers and running headers pasted mid-sentence. Search queries spanning these cuts also fail.Approach
For fixed-layout publications,
DefaultContentServicenow wraps the composite iterator in a newSentenceContentIterator, which re-segments the stream so that eachTextContentElementholds exactly one sentence:pageArtifact(extensible viaPageArtifactDetector), skipped by TTS and, by default, by search.HardBreakDetectorkeeps standalone display text out of sentences — "P A R T O N E", "Chapter 1" lines, title pages — emitted as their own spoken, searchable elements.continuedattribute, and each keeps a locator targeting its own page (page=fragment, cssSelector, interpolated PDF progression) — so decorations, page turns and "play from here" keep working per part.segment.textvalues concatenate to the spoken/logical form, while each locator'shighlightkeeps the on-page form ("particu-"). Emitted elements carrysentenceAlignedand pass throughmakeTextContentTokenizeruntouched.