Skip to content

Commit

Permalink
added preprocessing to exclude quotes
Browse files Browse the repository at this point in the history
  • Loading branch information
PriyaDCosta committed Apr 11, 2024
1 parent 0d0aa50 commit d57cd3c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion feature_engine/feature_builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ def preprocess_chat_data(self, col: str="message", turns=False, conversation_id=
# create new column that retains punctuation
self.chat_data["message_lower_with_punc"] = self.chat_data[col].astype(str).apply(preprocess_text_lowercase_but_retain_punctuation)

# create new column that removes any quoted text - for conflict
# create new column that removes any quoted text - for conflict
self.chat_data["message_without_quotes"] = self.chat_data[col].astype(str).apply(preprocess_remove_quotes_from_dataset)

# Preprocessing the text in `col` and then overwriting the column `col`.
Expand Down

0 comments on commit d57cd3c

Please sign in to comment.