Skip to content

in_separator in mwe_tokenize doesn't work properly #18

Open
@sinaahmadi

Description

The argument in_separator seems to be overwritten at the end of the mwe_tokenize function. The argument is not accessible through word_tokenizer either.

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions