Skip to content

Commit

Permalink
adding better chunking for txt2kg
Browse files Browse the repository at this point in the history
  • Loading branch information
riship committed Dec 10, 2024
1 parent 6c2a09f commit e15d908
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions torch_geometric/nn/nlp/txt2kg.py
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,9 @@ def get_num_procs():


def chunk_text(text: str, chunk_size: int = 512) -> list[str]:
"""Function to chunk text into sentence-based segments.
Co-authored with Claude AI
"""
Function to chunk text into sentence-based segments.
Co-authored with Claude AI.
"""
# If the input text is empty or None, return an empty list
if not text:
Expand Down

0 comments on commit e15d908

Please sign in to comment.