Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harrison/whatsapp loader #2085

Merged
merged 3 commits into from
Mar 28, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cr
  • Loading branch information
hwchase17 committed Mar 28, 2023
commit 912ad9477060161814efb42b5f5c42c74d7a4da2

This file was deleted.

67 changes: 0 additions & 67 deletions docs/modules/document_loaders/examples/whatsapp_chat.ipynb

This file was deleted.

1 change: 0 additions & 1 deletion langchain/document_loaders/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@
from langchain.document_loaders.url import UnstructuredURLLoader
from langchain.document_loaders.web_base import WebBaseLoader
from langchain.document_loaders.whatsapp_chat import WhatsAppChatLoader

from langchain.document_loaders.word_document import UnstructuredWordDocumentLoader
from langchain.document_loaders.youtube import (
GoogleApiClient,
Expand Down
5 changes: 4 additions & 1 deletion langchain/document_loaders/whatsapp_chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@
from langchain.docstore.document import Document
from langchain.document_loaders.base import BaseLoader


def concatenate_rows(date: str, sender: str, text: str) -> str:
"""Combine message information in a readable format ready to be used."""
return f"{sender} on {date}: {text}\n\n"


class WhatsAppChatLoader(BaseLoader):
"""Loader that loads WhatsApp messages text file."""

Expand All @@ -26,7 +28,8 @@ def load(self) -> List[Document]:

for line in lines:
result = re.match(
r"(\d{1,2}/\d{1,2}/\d{2,4}, \d{1,2}:\d{1,2} (?:AM|PM)) - (.*?): (.*)", line.strip()
r"(\d{1,2}/\d{1,2}/\d{2,4}, \d{1,2}:\d{1,2} (?:AM|PM)) - (.*?): (.*)",
line.strip(),
)
if result:
date, sender, text = result.groups()
Expand Down