Repository navigation
feat: preserve youtube transcript segments - #401
Open
lorenzozane wants to merge 1 commit into
Open
lorenzozane wants to merge 1 commit into
lorenzozane wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Possible implementation for #176
YoutubeExtractorcurrently groups transcript segments into readable blocks. This is useful for reading, but some projects could benefit from keeping the individual segments to have more granular timestamps.Added
extractors.youtube.preserveTranscriptSegments, defaulting tofalse, to make this optional.Example
The segments:
are currently grouped as:
With
preserveTranscriptSegments: true, each segment keeps its own timestamp:Implementation
Added
getTranscriptSegments()to either preserve the parsed segments or pass them through the existing grouping logic. Both fetched XML captions and transcripts extracted from the desktop/mobile DOM use this method.Text decoding, normalization and HTML escaping still apply. When preservation is enabled, speaker markers remain literal text and speaker turns are not inferred.
The option is also forwarded through synchronous extraction, asynchronous extraction and
fetchAsyncVariables(). Default behavior remains unchanged.Tests
Added coverage in
youtube-transcript.test.tsfor option forwarding, both XML formats, desktop/mobile transcripts, normalization, chapters and fractional start times. The existing HTML escaping test now runs with preservation both enabled and disabled.