Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing downloads when using different languages than English #27

Closed
goldyfruit opened this issue Nov 23, 2023 · 2 comments
Closed

Missing downloads when using different languages than English #27

goldyfruit opened this issue Nov 23, 2023 · 2 comments

Comments

@goldyfruit
Copy link
Member 

Nov 23 08:58:03 rpi3b ovos-core[3093]: 2023-11-23 08:58:03.320 - skill-ovos-wikipedia.openvoiceos - ERROR -
Nov 23 08:58:03 rpi3b ovos-core[3093]: **********************************************************************
Nov 23 08:58:03 rpi3b ovos-core[3093]:   Resource punkt not found.
Nov 23 08:58:03 rpi3b ovos-core[3093]:   Please use the NLTK Downloader to obtain the resource:
Nov 23 08:58:03 rpi3b ovos-core[3093]:   >>> import nltk
Nov 23 08:58:03 rpi3b ovos-core[3093]:   >>> nltk.download('punkt')
Nov 23 08:58:03 rpi3b ovos-core[3093]:   
Nov 23 08:58:03 rpi3b ovos-core[3093]:   For more information see: https://www.nltk.org/data.html
Nov 23 08:58:03 rpi3b ovos-core[3093]:   Attempted to load tokenizers/punkt/PY3/english.pickle
Nov 23 08:58:03 rpi3b ovos-core[3093]:   Searched in:
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/home/goldyfruit/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/home/goldyfruit/.venvs/ovos/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/home/goldyfruit/.venvs/ovos/share/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/home/goldyfruit/.venvs/ovos/lib/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/usr/share/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/usr/local/share/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/usr/lib/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/usr/local/lib/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - ''
Nov 23 08:58:03 rpi3b ovos-core[3093]: **********************************************************************
Nov 23 08:58:03 rpi3b ovos-core[3093]: 2023-11-23 08:58:03.327 - skills - ovos_workshop.skills.common_query_skill:__get_cq:160 - ERROR - error matching C'est quoi un reptile with skill-ovos-wikipedia.openvoiceos
Nov 23 08:58:03 rpi3b ovos-core[3093]: Traceback (most recent call last):
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/ovos_workshop/skills/common_query_skill.py", line 158, in __get_cq
Nov 23 08:58:03 rpi3b ovos-core[3093]:     result = self.CQS_match_query_phrase(search_phrase)
Nov 23 08:58:03 rpi3b ovos-core[3093]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 201, in CQS_match_query_phrase
Nov 23 08:58:03 rpi3b ovos-core[3093]:     title, summary = self.ask_the_wiki(phrase)
Nov 23 08:58:03 rpi3b ovos-core[3093]:                      ^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 227, in ask_the_wiki
Nov 23 08:58:03 rpi3b ovos-core[3093]:     self.image = self.wiki.get_image(query)
Nov 23 08:58:03 rpi3b ovos-core[3093]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 100, in get_image
Nov 23 08:58:03 rpi3b ovos-core[3093]:     data = self.extract_and_search(query, context)
Nov 23 08:58:03 rpi3b ovos-core[3093]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 55, in extract_and_search
Nov 23 08:58:03 rpi3b ovos-core[3093]:     query = self.extract_keyword(query, lang)
Nov 23 08:58:03 rpi3b ovos-core[3093]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 39, in extract_keyword
Nov 23 08:58:03 rpi3b ovos-core[3093]:     return keyword_extractor.extract_subject(query, lang)
Nov 23 08:58:03 rpi3b ovos-core[3093]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/ovos_classifiers/heuristics/keyword_extraction.py", line 58, in extract_subject
Nov 23 08:58:03 rpi3b ovos-core[3093]:     tagged_tokens = NltkPostag().tag(query)
Nov 23 08:58:03 rpi3b ovos-core[3093]:                     ^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/ovos_classifiers/heuristics/postag.py", line 149, in tag
Nov 23 08:58:03 rpi3b ovos-core[3093]:     sentence = nltk.word_tokenize(sentence)
Nov 23 08:58:03 rpi3b ovos-core[3093]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/nltk/tokenize/__init__.py", line 129, in word_tokenize
Nov 23 08:58:03 rpi3b ovos-core[3093]:     sentences = [text] if preserve_line else sent_tokenize(text, language)
Nov 23 08:58:03 rpi3b ovos-core[3093]:                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/nltk/tokenize/__init__.py", line 106, in sent_tokenize
Nov 23 08:58:03 rpi3b ovos-core[3093]:     tokenizer = load(f"tokenizers/punkt/{language}.pickle")
Nov 23 08:58:03 rpi3b ovos-core[3093]:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/nltk/data.py", line 750, in load
Nov 23 08:58:03 rpi3b ovos-core[3093]:     opened_resource = _open(resource_url)
Nov 23 08:58:03 rpi3b ovos-core[3093]:                       ^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/nltk/data.py", line 876, in _open
Nov 23 08:58:03 rpi3b ovos-core[3093]:     return find(path_, path + [""]).open()
Nov 23 08:58:03 rpi3b ovos-core[3093]:            ^^^^^^^^^^^^^^^^^^^^^^^^
Nov 23 08:58:03 rpi3b ovos-core[3093]:   File "/home/goldyfruit/.venvs/ovos/lib/python3.11/site-packages/nltk/data.py", line 583, in find
Nov 23 08:58:03 rpi3b ovos-core[3093]:     raise LookupError(resource_not_found)
Nov 23 08:58:03 rpi3b ovos-core[3093]: LookupError:
Nov 23 08:58:03 rpi3b ovos-core[3093]: **********************************************************************
Nov 23 08:58:03 rpi3b ovos-core[3093]:   Resource punkt not found.
Nov 23 08:58:03 rpi3b ovos-core[3093]:   Please use the NLTK Downloader to obtain the resource:
Nov 23 08:58:03 rpi3b ovos-core[3093]:   >>> import nltk
Nov 23 08:58:03 rpi3b ovos-core[3093]:   >>> nltk.download('punkt')
Nov 23 08:58:03 rpi3b ovos-core[3093]:   
Nov 23 08:58:03 rpi3b ovos-core[3093]:   For more information see: https://www.nltk.org/data.html
Nov 23 08:58:03 rpi3b ovos-core[3093]:   Attempted to load tokenizers/punkt/PY3/english.pickle
Nov 23 08:58:03 rpi3b ovos-core[3093]:   Searched in:
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/home/goldyfruit/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/home/goldyfruit/.venvs/ovos/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/home/goldyfruit/.venvs/ovos/share/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/home/goldyfruit/.venvs/ovos/lib/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/usr/share/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/usr/local/share/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/usr/lib/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - '/usr/local/lib/nltk_data'
Nov 23 08:58:03 rpi3b ovos-core[3093]:     - ''
Nov 23 08:58:03 rpi3b ovos-core[3093]: **********************************************************************

In order to fix the issue, I have to download manually the files.

nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')
nltk.download('universal_tagset')
@JarbasAl
Copy link
Member

JarbasAl commented Jan 4, 2024

duplicate of #32 ?

@goldyfruit
Copy link
Member Author

duplicate of #32 ?

It seems yes, got this in ovos_core but it seems to be the exact same issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@goldyfruit @JarbasAl