Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option case_sensitive to featurizers #5759

Closed
tabergma opened this issue Apr 30, 2020 · 0 comments · Fixed by #6147
Closed

Add option case_sensitive to featurizers #5759

tabergma opened this issue Apr 30, 2020 · 0 comments · Fixed by #6147
Assignees
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR

Comments

@tabergma
Copy link
Contributor

Description of Problem:
Currently, some of our tokenizers support the option case_sensitive. If the user decides to set this option to False all featurizer will use the lowercased tokens. This might not be ideal. For example, some of the features in the LexicalSyntacticFeaturizer do not work if all tokens are lowercased.
It might be better to move the option case_sensitive to the featurizers itself. Each featurizer can be configured separately.

Overview of the Solution:
Remove the option case_sensitive from all tokenizers.
Add the option case_sensitive to featurizers for which it makes sense.

@tabergma tabergma added type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR area:rasa-oss 🎡 Anything related to the open source Rasa framework labels Apr 30, 2020
@tabergma tabergma self-assigned this Jul 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant