Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileNotFoundError addons-7c.txt #55

Open
simahawk opened this issue Jul 14, 2021 · 47 comments
Open

FileNotFoundError addons-7c.txt #55

simahawk opened this issue Jul 14, 2021 · 47 comments

Comments

@simahawk
Copy link

simahawk commented Jul 14, 2021

Versions

Python: 3.8
setuptools-odoo: 2.7.1 (installed w/ pip3 install --user setuptools-odoo)
pre-commit: 2.13.0 (installed w/ pip3 install --user pre-commit)
Odoo version: 14.0

I get this when running pre-commit:

FileNotFoundError: [Errno 2] No such file or directory: '/home/sorsi/.cache/pre-commit/repo1mxdepkf/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/addons-7c.txt'

I find the file in /home/sorsi/.cache/pre-commit/repo1mxdepkf/setuptools_odoo/addons-7c.txt so it seems that the lookup for these txt files is broken here

Maybe it happens only w/ pre-commit + 3.8?

@sbidoul
Copy link
Member

sbidoul commented Jul 14, 2021

That's weird. Have you tried clearing your pre-commit cache ?

@simahawk
Copy link
Author

it's a brand new setup from scratch but I'll try!

@simahawk
Copy link
Author

@sbidoul tried again: no luck. I had to copy files there to make it work.

I'm in a virtualenv and I thought this was the issue but I've tried inside and outside it, no luck.

I've tried uninstalling venv installation of setuptools-odoo and pre-commit, no luck.

I've tried cleaning the cache in both cases, no luck.

# which pre-commit
/home/sorsi/.local/bin/pre-commit
# which setuptools-odoo-make-default 
/home/sorsi/.local/bin/setuptools-odoo-make-default

@sbidoul
Copy link
Member

sbidoul commented Jul 16, 2021

Can you share the relevant pre-commit config? I'll give it a try.

@sbidoul
Copy link
Member

sbidoul commented Jul 16, 2021

Note that which setuptools-odoo-make-default should be irrelevant, since pre-commit installs it and runs it in an isolated virtualenv.

@sbidoul
Copy link
Member

sbidoul commented Jul 16, 2021

Perhaps the fact that you installed both pre-commit and setuptools-odoo with pip install --user is confusing precommit ? FWIW, the only thing I personally install with --user is pipx, and then I pipx install pre-commit and all other tools from PyPI.

@sebalix
Copy link

sebalix commented Sep 8, 2021

I have the same issue, also on a fresh install/pre-commit cache, it says:

Generate default setup.py for an addons directory........................Failed
- hook id: setuptools-odoo-make-default
- exit code: 1

Traceback (most recent call last):
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/bin/setuptools-odoo-make-default", line 5, in <module>
    from setuptools_odoo.make_default_setup import main
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/__init__.py", line 6, in <module>
    from .core import get_addon_metadata
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/core.py", line 14, in <module>
    from . import base_addons, external_dependencies
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/base_addons.py", line 16, in <module>
    openerp7 = _addons("7c")
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/base_addons.py", line 12, in _addons
    b = resource_string("setuptools_odoo", "addons-%s.txt" % suffix)
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1142, in resource_string
    return get_provider(package_or_requirement).get_resource_string(
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1388, in get_resource_string
    return self._get(self._fn(self.module_path, resource_name))
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1611, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/addons-7c.txt'

pre-commit config: https://github.com/OCA/wms/blob/3d7446f47af2ec134ae14c6bc3d264b406d242cc/.pre-commit-config.yaml
And indeed pre-commit is installed with --user (but not setuptools-odoo, which is only installed in the pre-commit hook Python environment).

@sbidoul
Copy link
Member

sbidoul commented Sep 21, 2021

Can you try to create a reproducer in a container or something ?

@sbidoul
Copy link
Member

sbidoul commented May 6, 2022

Is this still an issue ?

@simahawk
Copy link
Author

simahawk commented May 9, 2022

Not AFAIK

@StefanRijnhart
Copy link

StefanRijnhart commented Jun 13, 2022

For the record, I' running into this. After clearing the cache and reinitializing pre-commit, I have the following versions:

(py3.8) $ . /home/odoo/.cache/pre-commit/repo4_14zsr6/py_env-python3/bin/activate
(py_env-python3) bruno@tulpa:~/emesa/oca/web$ pip list
Package         Version
--------------- -----------------
packaging       21.3
pip             22.1.2
pyparsing       3.0.9
setuptools      62.2.0
setuptools-odoo 0.1.dev1+gd744fc6
setuptools-scm  6.4.2
tomli           2.0.1
wheel           0.37.1

I can activate pre-commit's virtualenv and upgrade setuptools-odoo which fixes the problem, but it's not ideal.

@sbidoul
Copy link
Member

sbidoul commented Jun 13, 2022

@StefanRijnhart this looks like setuptools-odoo was installed from git or a git checkout ? That's weird. I can't really do much without a reproducer.

@simahawk simahawk reopened this Sep 16, 2022
@simahawk
Copy link
Author

@sbidoul I'm hitting this again on a setup from scratch.

Traceback (most recent call last):
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/bin/setuptools-odoo-make-default", line 5, in <module>
    from setuptools_odoo.make_default_setup import main
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/__init__.py", line 6, in <module>
    from .core import get_addon_metadata
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/core.py", line 14, in <module>
    from . import base_addons, external_dependencies
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/base_addons.py", line 16, in <module>
    openerp7 = _addons("7c")
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/base_addons.py", line 12, in _addons
    b = resource_string("setuptools_odoo", "addons-%s.txt" % suffix)
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1166, in resource_string
    return get_provider(package_or_requirement).get_resource_string(
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1412, in get_resource_string
    return self._get(self._fn(self.module_path, resource_name))
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1635, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/addons-7c.txt'

pre-commit 2.20.0

$ /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/bin/pip list
Package           Version
----------------- -----------------
packaging         21.3
pip               22.2.2
pyparsing         3.0.9
setuptools        65.1.1
setuptools-odoo   0.1.dev1+gd744fc6
setuptools-scm    7.0.5
tomli             2.0.1
typing_extensions 4.3.0
wheel             0.37.1
``

@simahawk
Copy link
Author

simahawk commented Sep 16, 2022

I confirm that upgrading setuptools-odoo fixes the problem

$ /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/bin/pip install setuptools-odoo
Requirement already satisfied: setuptools-odoo in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (0.1.dev1+gd744fc6)
Requirement already satisfied: setuptools-scm!=4.0.0,>=2.1 in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-odoo) (7.0.5)
Requirement already satisfied: setuptools in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-odoo) (65.1.1)
Requirement already satisfied: packaging>=20.0 in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-scm!=4.0.0,>=2.1->setuptools-odoo) (21.3)
Requirement already satisfied: tomli>=1.0.0 in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-scm!=4.0.0,>=2.1->setuptools-odoo) (2.0.1)
Requirement already satisfied: typing-extensions in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-scm!=4.0.0,>=2.1->setuptools-odoo) (4.3.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from packaging>=20.0->setuptools-scm!=4.0.0,>=2.1->setuptools-odoo) (3.0.9)
(cos14) $ sorsi@pop-os (14-edi-drop-ack-create *+) ~/dev/projects/cosanum_odoo/odoo/external-src/edi 
$ /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/bin/pip install setuptools-odoo --upgrade
[...]
Installing collected packages: setuptools-odoo
  Attempting uninstall: setuptools-odoo
    Found existing installation: setuptools-odoo 0.1.dev1+gd744fc6
    Uninstalling setuptools-odoo-0.1.dev1+gd744fc6:
      Successfully uninstalled setuptools-odoo-0.1.dev1+gd744fc6
Successfully installed setuptools-odoo-3.1.4

So for some reasons, pre-commit installs 0.1.dev1+gd744fc6 instead of the latest version 🤔

pre-commit config https://github.com/OCA/edi/blob/14.0/.pre-commit-config.yaml

@sbidoul
Copy link
Member

sbidoul commented Sep 16, 2022

This setuptools-odoo 0.1.dev1+gd744fc6 is not on PyPI, so you'll have to find out how it ended up in your pre-commit env.

@simahawk
Copy link
Author

I know but I have no clue 😅
I'm working on a brand new laptop where I had to re-setup from scratch my projects' envs.
Here I just ran pre-commit install and committed something.
Also, here we have no control on how setuptools-odoo is installed because is done by pre-commit 🤷‍♂️

@sbidoul
Copy link
Member

sbidoul commented Sep 16, 2022

Commit d744fc6 (of 0.1.dev1+gd744fc6) corresponds to the 2.6.0 tag.

@sbidoul
Copy link
Member

sbidoul commented Sep 16, 2022

Could you try manually reinstalling setuptools-odoo==2.6.0 in the pre-commit generated venv?

@sbidoul
Copy link
Member

sbidoul commented Sep 16, 2022

The fact that it shows version 0.1.dev1+gd744fc6 could be because pre-commit does not fetch tags, but that should not prevent setuptools to work normally.

@sbidoul
Copy link
Member

sbidoul commented Sep 16, 2022

If a repro situation, can you also list the content of {thevenv}/python3.10/site-packages/setuptools_odoo ?

@max3903
Copy link

max3903 commented Oct 10, 2022

Same problem here.

@sbidoul
Copy link
Member

sbidoul commented Oct 10, 2022

Installing pre-commit with pipx and emptying the pre-commit cache fixed the issue.

It is likely due to some incompatibility with the system setuptools.

@dreispt
Copy link
Contributor

dreispt commented Nov 15, 2022

Getting the same problem, after running copier update on an OCA repo.
As a workaround, I'm setting the setuptools-odoo version back to 2.6.0.

@sbidoul
Copy link
Member

sbidoul commented Nov 15, 2022

The best explanation we have is an incompatibility somewhere between pre-commit, setuptools_scm and the system python packages.

The recommendation is to install pre-commit using pipx.

@dreispt
Copy link
Contributor

dreispt commented Nov 16, 2022

I actually already had pre-commit with pipx.
I ran:

pip uninstall pre-commit
pipx uninstall pre-commit
pipx install pre-commit

Still get same error.

Not sure if this helps:

$ pipx list
venvs are in /home/daniel/.local/pipx/venvs
apps are exposed on your $PATH at /home/daniel/.local/bin
   package copier 5.1.0, installed using Python 3.10.4
    - copier (symlink missing or pointing to unexpected location)
   package pre-commit 2.20.0, installed using Python 3.10.6
    - pre-commit
    - pre-commit-validate-config
    - pre-commit-validate-manifest

@dreispt
Copy link
Contributor

dreispt commented Nov 16, 2022

Addition info - from trial and error, it stops working for me at setuptools-odoo 3.1.5.
Last version that works is 3.1.4.

@sbidoul
Copy link
Member

sbidoul commented Nov 16, 2022

Addition info - from trial and error, it stops working for me at setuptools-odoo 3.1.5.
Last version that works is 3.1.4.

That is weird. The diff between these two version has really nothing that could explain that.

@dreispt
Copy link
Contributor

dreispt commented Nov 16, 2022

Some imports changed. Maybe that activated code that triggers the issue?

@dreispt
Copy link
Contributor

dreispt commented Nov 16, 2022

I dug into the virtualenv being used: activated it, ran setuptools-odoo-make-default, and saw the error happening.
The error happens at pkg_resources/__init__.py +1636, running with open(path, 'rb') as stream:
The value in path is .../site-packages/setuptools_odoo/addons-7c.txt.
But no txt files exist there - the directory content is:

base_addons.py  external_dependencies.py  git_postversion.py  make_default_setup.py  __pycache__
core.py         get_requirements.py       __init__.py         manifest.py            setup_keywords.py

Can this be a packaging issue?

@dreispt
Copy link
Contributor

dreispt commented Nov 16, 2022

In case it helps, venv package list:

$ pip list
Package           Version
----------------- -----------------
packaging         21.3
pip               22.3.1
pyparsing         3.0.9
setuptools        65.5.1
setuptools-odoo   0.1.dev1+gecdc64b
setuptools-scm    7.0.5
tomli             2.0.1
typing_extensions 4.4.0
wheel             0.37.1

@sbidoul
Copy link
Member

sbidoul commented Nov 16, 2022

Some imports changed.

I'm confused: this is the diff I see between 3.1.4 and 3.1.5

image

@sbidoul
Copy link
Member

sbidoul commented Nov 16, 2022

Again, I'd be happy to investigate but I can't reproduce. If someone can reproduce in a Dockerfile or something I'll be happy to dig into it.

@dreispt
Copy link
Contributor

dreispt commented Nov 16, 2022

Found a solution that worked for me - turn off your computer and turn it on again ... or almost that 😸 :
Just run pre-commit clean, and after that pre-commit run -a worked fine.

@simahawk
Copy link
Author

Found a solution that worked for me - turn off your computer and turn it on again ... or almost that smile_cat : Just run pre-commit clean, and after that pre-commit run -a worked fine.

pre-commit clean does not fix anything for me 😭

I just did this:

  1. uninstalled pre-commit from my pyenv / virtualenv
  2. installed it from scratch usin pipx
  3. ran pre-commit clean
  4. run again -> same error 🤷‍♂️

@simahawk
Copy link
Author

FileNotFoundError: [Errno 2] No such file or directory: '/home/sorsi/.cache/pre-commit/repope1kgfmy/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/addons-7c.txt'

$ find -name addons-7c.txt
./repope1kgfmy/setuptools_odoo/addons-7c.txt

The issue seems to be that somehow the file gets located in the wrong place...

@simahawk
Copy link
Author

@sbidoul BTW I'm on v14, what's the point of loading all the versions? (I'm checking)

@simahawk
Copy link
Author

$ find -name setuptools_odoo
./pre-commit/repod6fj5j15/setuptools_odoo
./pre-commit/repod6fj5j15/build/lib/setuptools_odoo
./pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/setuptools_odoo
$ tree ./pre-commit/repod6fj5j15/setuptools_odoo
./pre-commit/repod6fj5j15/setuptools_odoo
├── addons-10c.txt
├── addons-10e.txt
├── addons-11c.txt
├── addons-11e.txt
├── addons-12c.txt
├── addons-12e.txt
├── addons-13c.txt
├── addons-13e.txt
├── addons-14c.txt
├── addons-14e.txt
├── addons-15c.txt
├── addons-15e.txt
├── addons-16c.txt
├── addons-16e.txt
├── addons-7c.txt
├── addons-8c.txt
├── addons-9c.txt
├── addons-9e.txt
├── base_addons.py
├── core.py
├── external_dependencies.py
├── get_requirements.py
├── git_postversion.py
├── __init__.py
├── make_default_setup.py
├── manifest.py
└── setup_keywords.py
$ tree ./pre-commit/repod6fj5j15/build/lib/setuptools_odoo/
./pre-commit/repod6fj5j15/build/lib/setuptools_odoo/
├── base_addons.py
├── core.py
├── external_dependencies.py
├── get_requirements.py
├── git_postversion.py
├── __init__.py
├── make_default_setup.py
├── manifest.py
└── setup_keywords.py

$ tree ./pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/setuptools_odoo
./pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/setuptools_odoo
├── base_addons.py
├── core.py
├── external_dependencies.py
├── get_requirements.py
├── git_postversion.py
├── __init__.py
├── make_default_setup.py
├── manifest.py
├── __pycache__
│   ├── base_addons.cpython-310.pyc
│   ├── core.cpython-310.pyc
│   ├── external_dependencies.cpython-310.pyc
│   ├── get_requirements.cpython-310.pyc
│   ├── git_postversion.cpython-310.pyc
│   ├── __init__.cpython-310.pyc
│   ├── make_default_setup.cpython-310.pyc
│   ├── manifest.cpython-310.pyc
│   └── setup_keywords.cpython-310.pyc
└── setup_keywords.py

This looks insane.

@simahawk
Copy link
Author

> /home/sorsi/.cache/pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py(351)get_provider()
-> def get_provider(moduleOrReq):
> /home/sorsi/.cache/pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py(1411)

1411  ->     def get_resource_string(self, manager, resource_name):                                                                                                                      
1412             return self._get(self._fn(self.module_path, resource_name))                                                                                                             
(Pdb++) self
<pkg_resources.DefaultProvider object at 0x7f33dd141630>
(Pdb++) self.module_path
'/home/sorsi/.cache/pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/setuptools_odoo'

As you see, inside the pkg_resource lookup, the module path is the latter, which is missing those files.

@dreispt
Copy link
Contributor

dreispt commented Nov 19, 2022

@simahawk Sharing my result:

$ find -name addons-7c.txt
./repohr54hezv/build/lib/setuptools_odoo/addons-7c.txt
./repohr54hezv/setuptools_odoo/addons-7c.txt
./repohr54hezv/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/addons-7c.txt
./repobiz5uwfh/build/lib/setuptools_odoo/addons-7c.txt
./repobiz5uwfh/setuptools_odoo/addons-7c.txt
./repobiz5uwfh/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/addons-7c.txt
./repo469mamt8/setuptools_odoo/addons-7c.txt
./repof8w2ztgf/setuptools_odoo/addons-7c.txt

I have four caches, all created recently (because I recently ran pre-commit clean).
In two of them I see the same result as you, in two others I see the file also in lib and py_env.

I have no idea if this is helpful...

@simahawk
Copy link
Author

simahawk commented Jun 1, 2023

@sbidoul once again getting back to this... Here's a brand new setup cleaned w/ pre-commit clean:

setuptools-broken-txt-position

After my last cp command it's solved.
See also #55 (comment)

@sbidoul
Copy link
Member

sbidoul commented Jun 1, 2023

@simahawk could you do a Dockerfile with a reproducer?

@simahawk
Copy link
Author

simahawk commented Jun 1, 2023

I'll try, not sure when 😅

@cuongnmtm
Copy link

cuongnmtm commented Jul 27, 2023

I am facing this issue and trying to debug.

There is no data file in the cache

image

So I try to check why data files are not copied when setuptools try to setup the package. I checked the code and it looks good with include_package_data=True in setuptools.setup().

Then I research on the internet and found this https://stackoverflow.com/questions/7522250/how-to-include-package-data-with-setuptools-distutils/14159430#14159430
I am not sure the answer is related to this issue or not, but gave it a try.
I added include addons*.txt to MANIFEST.in and it seems it solved my issue

image

@sbidoul
Copy link
Member

sbidoul commented Aug 1, 2023

@cuongnmtm since setuptools-odoo uses the git file finder via setuptools_scm, updating MANIFEST.in should not be necessary.

By chance do you have a procedure to reproduce the issue?

@cuongnmtm
Copy link

@sbidoul I just tried to reproduce the issue but cannot...

I tried to run pre-commit clean and pre-commit run -a, no issue now. But at the time I faced the issue, this didn't solve the issue.

This only happens with very few of my projects (actually just one of two projects). Let's see if I faced this issue again in the future.

Thank you.

@SirAionTech
Copy link

I faced this exact issue about file "addons-7c.txt" and another one about the readme generation, that complained about a missing "gen_addon_readme.rst.jinja" somewhere.

Re-installing pre-commit fixed it for me:

pre-commit clean
pipx uninstall pre-commit
pipx install pre-commit

Now pre-commit run --all-files works perfectly.

Hope this helps 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants