Hotfix add htmlcov to flat layout package finder.default exclude #3836

yukihiko-shinoda · 2023-02-23T07:42:57Z

Summary of changes

Added htmlcov to FlatLayoutPackageFinder.DEFAULT_EXCLUDE.

#3715 still remains issue:

The command: `pip install --editable` still fails:

# pip install -e .
Obtaining file:///workspace
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      /tmp/pip-build-env-fyn78agv/overlay/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
        warnings.warn(msg, _BetaConfiguration)
      error: Multiple top-level packages discovered in a flat-layout: ['htmlcov', 'examplepackage'].
      
      To avoid accidental inclusion of unwanted files or directories,
      setuptools will not proceed with this build.
      
      If you are trying to create a single distribution with multiple packages
      on purpose, you should not rely on automatic discovery.
      Instead, consider the following options:
      
      1. set up custom discovery (`find` directive with `include` or `exclude`)
      2. use a `src-layout`
      3. explicitly set `py_modules` or `packages` with a list of names
      
      To find more information, look for "package discovery" on setuptools docs.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Integrated test succeed by using this branch:

[build-system]
requires = ["setuptools@git+https://github.com/yukihiko-shinoda/setuptools.git@hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE"]
build-backend = "setuptools.build_meta"

The command: `pip install --editable` is fixed:

# pip install -e . --verbose
Using pip 23.0.1 from /usr/local/lib/python3.11/site-packages/pip (python 3.11)
Obtaining file:///workspace
  Running command pip subprocess to install build dependencies
  Collecting setuptools@ git+https://github.com/yukihiko-shinoda/setuptools.git@hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE
    Cloning https://github.com/yukihiko-shinoda/setuptools.git (to revision hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE) to /tmp/pip-install-p2og1wji/setuptools_20e1fd3a7e9740408f23b3e39e39728a
    Running command git clone --filter=blob:none --quiet https://github.com/yukihiko-shinoda/setuptools.git /tmp/pip-install-p2og1wji/setuptools_20e1fd3a7e9740408f23b3e39e39728a
    Running command git checkout -b hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE --track origin/hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE
    Switched to a new branch 'hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE'
    Branch 'hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE' set up to track remote branch 'hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE' from 'origin'.
    Resolved https://github.com/yukihiko-shinoda/setuptools.git to commit 9182fef90589676c3f361d65d457ea97d4b9510f
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Installing backend dependencies: started
    Installing backend dependencies: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: setuptools
    Building wheel for setuptools (pyproject.toml): started
    Building wheel for setuptools (pyproject.toml): finished with status 'done'
    Created wheel for setuptools: filename=setuptools-67.4.0.post20230223-py3-none-any.whl size=1088479 sha256=ae98e1ca4bd02cabc0d3794fd4674265f901a3565c13445c1a67a7ff05b17f0f
    Stored in directory: /tmp/pip-ephem-wheel-cache-d19brlch/wheels/87/2c/5d/b1b0e64626e12f658fc5d950e2f7ea981092d18a9cca625b98
  Successfully built setuptools
  Installing collected packages: setuptools
  Successfully installed setuptools-67.4.0.post20230223
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  Installing build dependencies ... done
  Running command Checking if build backend supports build_editable
  Checking if build backend supports build_editable ... done
  Running command Getting requirements to build editable
  /tmp/pip-build-env-wlh43ph8/overlay/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
    warnings.warn(msg, _BetaConfiguration)
  running egg_info
  writing examplepackage.egg-info/PKG-INFO
  writing dependency_links to examplepackage.egg-info/dependency_links.txt
  writing requirements to examplepackage.egg-info/requires.txt
  writing top-level names to examplepackage.egg-info/top_level.txt
  reading manifest file 'examplepackage.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file 'examplepackage.egg-info/SOURCES.txt'
  Getting requirements to build editable ... done
  Running command pip subprocess to install backend dependencies
  Collecting wheel
    Using cached wheel-0.38.4-py3-none-any.whl (36 kB)
  Installing collected packages: wheel
  Successfully installed wheel-0.38.4
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  Installing backend dependencies ... done
  Running command Preparing editable metadata (pyproject.toml)
  /tmp/pip-build-env-wlh43ph8/overlay/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
    warnings.warn(msg, _BetaConfiguration)
  running dist_info
  creating /tmp/pip-modern-metadata-0zjedxyb/examplepackage.egg-info
  writing /tmp/pip-modern-metadata-0zjedxyb/examplepackage.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-modern-metadata-0zjedxyb/examplepackage.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-modern-metadata-0zjedxyb/examplepackage.egg-info/requires.txt
  writing top-level names to /tmp/pip-modern-metadata-0zjedxyb/examplepackage.egg-info/top_level.txt
  writing manifest file '/tmp/pip-modern-metadata-0zjedxyb/examplepackage.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-modern-metadata-0zjedxyb/examplepackage.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-modern-metadata-0zjedxyb/examplepackage.egg-info/SOURCES.txt'
  creating '/tmp/pip-modern-metadata-0zjedxyb/examplepackage-0.1.0.dist-info'
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: examplepackage
  Running command Building editable for examplepackage (pyproject.toml)
  /tmp/pip-build-env-wlh43ph8/overlay/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
    warnings.warn(msg, _BetaConfiguration)
  running editable_wheel
  creating /tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage.egg-info
  writing /tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage.egg-info/requires.txt
  writing top-level names to /tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage.egg-info/top_level.txt
  writing manifest file '/tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage.egg-info/SOURCES.txt'
  creating '/tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage-0.1.0.dist-info'
  creating /tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage-0.1.0.dist-info/WHEEL
  running build_py
  running egg_info
  creating /tmp/tmp61v2pjih.build-temp/examplepackage.egg-info
  writing /tmp/tmp61v2pjih.build-temp/examplepackage.egg-info/PKG-INFO
  writing dependency_links to /tmp/tmp61v2pjih.build-temp/examplepackage.egg-info/dependency_links.txt
  writing requirements to /tmp/tmp61v2pjih.build-temp/examplepackage.egg-info/requires.txt
  writing top-level names to /tmp/tmp61v2pjih.build-temp/examplepackage.egg-info/top_level.txt
  writing manifest file '/tmp/tmp61v2pjih.build-temp/examplepackage.egg-info/SOURCES.txt'
  reading manifest file '/tmp/tmp61v2pjih.build-temp/examplepackage.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file '/tmp/tmp61v2pjih.build-temp/examplepackage.egg-info/SOURCES.txt'
  Editable install will be performed using a meta path finder.

  Options like `package-data`, `include/exclude-package-data` or
  `packages.find.exclude/include` may have no effect.

  adding '__editable___examplepackage_0_1_0_finder.py'
  adding '__editable__.examplepackage-0.1.0.pth'
  creating '/tmp/pip-wheel-lo8mljie/.tmp-rirdxxj6/examplepackage-0.1.0-0.editable-py3-none-any.whl' and adding '/tmp/tmpgs9yglkeexamplepackage-0.1.0-0.editable-py3-none-any.whl' to it
  adding 'examplepackage-0.1.0.dist-info/LICENSE'
  adding 'examplepackage-0.1.0.dist-info/METADATA'
  adding 'examplepackage-0.1.0.dist-info/WHEEL'
  adding 'examplepackage-0.1.0.dist-info/top_level.txt'
  adding 'examplepackage-0.1.0.dist-info/RECORD'
  /tmp/pip-build-env-wlh43ph8/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py:508: InformationOnly:

          Please be careful with folders in your working directory with the same
          name as your package as they may take precedence during imports.

    warnings.warn(msg, InformationOnly)
  Building editable for examplepackage (pyproject.toml) ... done
  Created wheel for examplepackage: filename=examplepackage-0.1.0-0.editable-py3-none-any.whl size=4487 sha256=1fbe74bbb68d1aef17bb0bb4ff5f0fd338c0ec0c6431173918023e5d88069873
  Stored in directory: /tmp/pip-ephem-wheel-cache-vvnbmyp4/wheels/e2/2c/68/4c4a53c0aec7bc75cfaa7bb42a2542d1390caa1e3199ccef70
Successfully built examplepackage
Installing collected packages: examplepackage
Successfully installed examplepackage-0.1.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@b450ea04586e:/workspace# pip list
Package          Version   Editable project location
---------------- --------- -------------------------
certifi          2022.12.7
distlib          0.3.6
filelock         3.9.0
examplepackage   0.1.0     /workspace
pip              23.0.1
pipenv           2023.2.18
platformdirs     3.0.0
setuptools       67.3.3
virtualenv       20.19.0
virtualenv-clone 0.5.7
wheel            0.38.4
root@b450ea04586e:/workspace#

Pull Request Checklist

Changes have tests
News fragment added in changelog.d/.
(See documentation for details)

abravalheri · 2023-02-24T14:29:52Z

Thank you very much for the contribution @yukihiko-shinoda. I think we can queue the changes for the next release.

Is it correct to assume that htmlcov is always a folder? Does it make sense to keep it in FlatLayoutModuleFinder?

yukihiko-shinoda · 2023-02-25T05:00:11Z

Hi @abravalheri , I searched official doc and GitHub code of latest version by keyword: htmlcov, the htmlcov seems to be always defined as not file but directory to put index.html for code coverage.

I can't judge whether it make sense or not to keep it in FlatLayoutModuleFinder, bacause:

I'm stranger for specification of FlatLayoutModuleFinder and setuptools
I don't know how Add htmlcov to FlatLayoutModuleFinder.DEFAULT_EXCLUDE #3715 test and which case it assume.

However, I tryed to remove htmlcov entry from FlatLayoutModuleFinder, dumped coverage into htmlcov directory and:

Tested pip install --editable ., it worked fine.
The following code examplepackage/test.py seems to work file:

from setuptools.discovery import ConfigDiscovery
from setuptools.dist import Distribution

config_discovery = ConfigDiscovery(Distribution())
config_discovery()
print(config_discovery.dist.packages)
print(config_discovery.dist.py_modules)

Results:

$ python examplepackage/test.py
['examplepackage']
None

It seems that it was previously added there by mistake. After some inspection ``htmlcov`` should be a folder (by default) and therefore can only be confused by a package (not a module)

abravalheri · 2023-02-27T10:48:15Z

Thank you very much @yukihiko-shinoda.

I think it is safe to remove it from FlatLayoutModuleFinder.
I added 2 commits addressing that:

89e7f64 - Remove the confusing htmlcov entry in FlatLayoutModuleFinder
46db21a - I changed the news fragment to also reflect the removal and I renamed the file so setuptools automatic versioning will classify this PR as a patch level version bump.

I hope these changes are OK, please let me know if you disagree, otherwise I might add this to a release later this week or the next one.

yukihiko-shinoda added 3 commits February 23, 2023 16:22

Add FlatLayoutPackageFinder's test for Coverage.py

4bce9df

Hotfix add htmlcov to FlatLayoutPackageFinder.DEFAULT_EXCLUDE

17aa0a6

Add changelog entry

9bfb5bf

Remove htmlcov from auto **module** discovery

89e7f64

It seems that it was previously added there by mistake. After some inspection ``htmlcov`` should be a folder (by default) and therefore can only be confused by a package (not a module)

Adjust news fragment for more descriptiveness

46db21a

abravalheri force-pushed the hotfix-add-htmlcov-to-FlatLayoutPackageFinder.DEFAULT_EXCLUDE branch from 64e1f06 to 46db21a Compare February 27, 2023 10:49

abravalheri merged commit 865d790 into pypa:main Mar 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hotfix add htmlcov to flat layout package finder.default exclude #3836

Hotfix add htmlcov to flat layout package finder.default exclude #3836

yukihiko-shinoda commented Feb 23, 2023

abravalheri commented Feb 24, 2023

yukihiko-shinoda commented Feb 25, 2023 •

edited

Loading

abravalheri commented Feb 27, 2023 •

edited

Loading

Hotfix add htmlcov to flat layout package finder.default exclude #3836

Hotfix add htmlcov to flat layout package finder.default exclude #3836

Conversation

yukihiko-shinoda commented Feb 23, 2023

Summary of changes

Pull Request Checklist

abravalheri commented Feb 24, 2023

yukihiko-shinoda commented Feb 25, 2023 • edited Loading

abravalheri commented Feb 27, 2023 • edited Loading

yukihiko-shinoda commented Feb 25, 2023 •

edited

Loading

abravalheri commented Feb 27, 2023 •

edited

Loading