Description
🐛 Bugs / Unexpected behaviors
Versions
Python: 3.8.10
OS: Ubuntu 20.04.5 LTS
Poetry: 1.6.1
pip: 23.2.1
Description
I am trying to set up a poetry environment with this library installed. I started off with a completely new, empty environment. Then I attempted to add the poetry3d
package:
poetry add -vvv git+https://github.com/facebookresearch/pytorch3d.git
Instead of adding it to the poetry environment, I got the following error:
Cloning https://github.com/facebookresearch/pytorch3d.git at 'HEAD' to /home/myfolder/poetry_env/poetry_env_demo/.venv/src/pytorch3d
Stack trace:
1 ~/.local/lib/python3.8/site-packages/poetry/utils/env/base_env.py:344 in _run
342│ try:
343│ if input_:
→ 344│ output: str = subprocess.run(
345│ cmd,
346│ stdout=subprocess.PIPE,
CalledProcessError
Command '['/tmp/tmpxdbpwgq0/.venv/bin/python', '-']' returned non-zero exit status 1.
at /usr/lib/python3.8/subprocess.py:516 in run
512│ # We don't call process.wait() as .__exit__ does that for us.
513│ raise
514│ retcode = process.poll()
515│ if check and retcode:
→ 516│ raise CalledProcessError(retcode, process.args,
517│ output=stdout, stderr=stderr)
518│ return CompletedProcess(process.args, retcode, stdout, stderr)
519│
520│
The following error occurred when trying to handle this error:
Stack trace:
3 ~/.local/lib/python3.8/site-packages/poetry/inspection/info.py:601 in get_pep517_metadata
599│ *PEP517_META_BUILD_DEPS,
600│ )
→ 601│ venv.run(
602│ "python",
603│ "-",
2 ~/.local/lib/python3.8/site-packages/poetry/utils/env/base_env.py:314 in run
312│ def run(self, bin: str, *args: str, **kwargs: Any) -> str:
313│ cmd = self.get_command_from_bin(bin) + list(args)
→ 314│ return self._run(cmd, **kwargs)
315│
316│ def run_pip(self, *args: str, **kwargs: Any) -> str:
1 ~/.local/lib/python3.8/site-packages/poetry/utils/env/virtual_env.py:96 in _run
94│ def _run(self, cmd: list[str], **kwargs: Any) -> str:
95│ kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
→ 96│ return super()._run(cmd, **kwargs)
97│
98│ def get_temp_environ(
EnvCommandError
Command ['/tmp/tmpxdbpwgq0/.venv/bin/python', '-'] errored with the following return code 1
Output:
Traceback (most recent call last):
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 466, in _handle_backend
yield
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 356, in get_requires_for_build
return set(get_requires(config_settings))
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
return self._call_hook('get_requires_for_build_wheel', {
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/pyproject_hooks/_impl.py", line 311, in _call_hook
self._subprocess_runner(
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 302, in _runner
self._hook_runner(cmd, cwd, extra_environ)
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/pyproject_hooks/_impl.py", line 71, in quiet_subprocess_runner
check_output(cmd, cwd=cwd, env=env, stderr=STDOUT)
File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/build-env-8vw3hcw7/bin/python', '/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmp9cn0rpj_']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 16, in <module>
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 356, in get_requires_for_build
return set(get_requires(config_settings))
File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 474, in _handle_backend
raise BuildBackendException( # noqa: B904 # use raise from
build.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel
Input:
import build
import build.env
import pyproject_hooks
source = '/home/myfolder/poetry_env/poetry_env_demo/.venv/src/pytorch3d'
dest = '/tmp/tmpxdbpwgq0/dist'
with build.env.IsolatedEnvBuilder() as env:
builder = build.ProjectBuilder(
srcdir=source,
scripts_dir=env.scripts_dir,
python_executable=env.executable,
runner=pyproject_hooks.quiet_subprocess_runner,
)
env.install(builder.build_system_requires)
env.install(builder.get_requires_for_build('wheel'))
builder.metadata_path(dest)
at ~/.local/lib/python3.8/site-packages/poetry/utils/env/base_env.py:363 in _run
359│ output = subprocess.check_output(
360│ cmd, stderr=stderr, env=env, text=True, **kwargs
361│ )
362│ except CalledProcessError as e:
→ 363│ raise EnvCommandError(e, input=input_)
364│
365│ return output
366│
367│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
The following error occurred when trying to handle this error:
Stack trace:
2 ~/.local/lib/python3.8/site-packages/poetry/utils/env/base_env.py:359 in _run
357│ output = ""
358│ else:
→ 359│ output = subprocess.check_output(
360│ cmd, stderr=stderr, env=env, text=True, **kwargs
361│ )
1 /usr/lib/python3.8/subprocess.py:415 in check_output
413│ kwargs['input'] = empty
414│
→ 415│ return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
416│ **kwargs).stdout
417│
CalledProcessError
Command '['/tmp/tmpxdbpwgq0/.venv/bin/python', 'setup.py', 'egg_info']' returned non-zero exit status 1.
at /usr/lib/python3.8/subprocess.py:516 in run
512│ # We don't call process.wait() as .__exit__ does that for us.
513│ raise
514│ retcode = process.poll()
515│ if check and retcode:
→ 516│ raise CalledProcessError(retcode, process.args,
517│ output=stdout, stderr=stderr)
518│ return CompletedProcess(process.args, retcode, stdout, stderr)
519│
520│
The following error occurred when trying to handle this error:
Stack trace:
3 ~/.local/lib/python3.8/site-packages/poetry/inspection/info.py:622 in get_pep517_metadata
620│ os.chdir(path)
621│ try:
→ 622│ venv.run("python", "setup.py", "egg_info")
623│ info = PackageInfo.from_metadata(path)
624│ except EnvCommandError as fbe:
2 ~/.local/lib/python3.8/site-packages/poetry/utils/env/base_env.py:314 in run
312│ def run(self, bin: str, *args: str, **kwargs: Any) -> str:
313│ cmd = self.get_command_from_bin(bin) + list(args)
→ 314│ return self._run(cmd, **kwargs)
315│
316│ def run_pip(self, *args: str, **kwargs: Any) -> str:
1 ~/.local/lib/python3.8/site-packages/poetry/utils/env/virtual_env.py:96 in _run
94│ def _run(self, cmd: list[str], **kwargs: Any) -> str:
95│ kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
→ 96│ return super()._run(cmd, **kwargs)
97│
98│ def get_temp_environ(
EnvCommandError
Command ['/tmp/tmpxdbpwgq0/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1
Output:
Traceback (most recent call last):
File "setup.py", line 15, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
at ~/.local/lib/python3.8/site-packages/poetry/utils/env/base_env.py:363 in _run
359│ output = subprocess.check_output(
360│ cmd, stderr=stderr, env=env, text=True, **kwargs
361│ )
362│ except CalledProcessError as e:
→ 363│ raise EnvCommandError(e, input=input_)
364│
365│ return output
366│
367│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
The following error occurred when trying to handle this error:
Stack trace:
18 ~/.local/lib/python3.8/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a
17 ~/.local/lib/python3.8/site-packages/poetry/console/application.py:190 in _run
188│ self._load_plugins(io)
189│
→ 190│ exit_code: int = super()._run(io)
191│ return exit_code
192│
16 ~/.local/lib/python3.8/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│
15 ~/.local/lib/python3.8/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code
14 ~/.local/lib/python3.8/site-packages/cleo/application.py:457 in _run_command
455│
456│ if command_event.command_should_run():
→ 457│ exit_code = command.run(io)
458│ else:
459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
13 ~/.local/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:
12 ~/.local/lib/python3.8/site-packages/cleo/commands/command.py:62 in execute
60│
61│ try:
→ 62│ return self.handle()
63│ except KeyboardInterrupt:
64│ return 1
11 ~/.local/lib/python3.8/site-packages/poetry/console/commands/add.py:160 in handle
158│ return 0
159│
→ 160│ requirements = self._determine_requirements(
161│ packages,
162│ allow_prereleases=self.option("allow-prereleases"),
10 ~/.local/lib/python3.8/site-packages/poetry/console/commands/init.py:375 in _determine_requirements
373│
374│ result = []
→ 375│ for requirement in self._parse_requirements(requires):
376│ if "git" in requirement or "url" in requirement or "path" in requirement:
377│ result.append(requirement)
9 ~/.local/lib/python3.8/site-packages/poetry/console/commands/init.py:441 in _parse_requirements
439│ cwd=cwd,
440│ )
→ 441│ return [parser.parse(requirement) for requirement in requirements]
442│
443│ def _format_requirements(self, requirements: list[dict[str, str]]) -> Requirements:
8 ~/.local/lib/python3.8/site-packages/poetry/console/commands/init.py:441 in <listcomp>
439│ cwd=cwd,
440│ )
→ 441│ return [parser.parse(requirement) for requirement in requirements]
442│
443│ def _format_requirements(self, requirements: list[dict[str, str]]) -> Requirements:
7 ~/.local/lib/python3.8/site-packages/poetry/utils/dependency_specification.py:89 in parse
87│
88│ specification = (
→ 89│ self._parse_url(requirement)
90│ or self._parse_path(requirement)
91│ or self._parse_simple(requirement)
6 ~/.local/lib/python3.8/site-packages/poetry/utils/dependency_specification.py:149 in _parse_url
147│
148│ if url_parsed.scheme in GIT_URL_SCHEMES:
→ 149│ return self._parse_git_url(requirement)
150│
151│ if url_parsed.scheme in ["http", "https"]:
5 ~/.local/lib/python3.8/site-packages/poetry/utils/dependency_specification.py:133 in _parse_git_url
131│
132│ source_root = self._env.path.joinpath("src") if self._env else None
→ 133│ package = self._direct_origin.get_package_from_vcs(
134│ "git",
135│ url=url.url,
4 ~/.local/lib/python3.8/site-packages/poetry/packages/direct_origin.py:111 in get_package_from_vcs
109│ raise ValueError(f"Unsupported VCS dependency {vcs}")
110│
→ 111│ return _get_package_from_git(
112│ url=url,
113│ branch=branch,
3 ~/.local/lib/python3.8/site-packages/poetry/packages/direct_origin.py:46 in _get_package_from_git
44│ path = path.joinpath(subdirectory)
45│
→ 46│ package = DirectOrigin.get_package_from_directory(path)
47│ package._source_type = "git"
48│ package._source_url = url
2 ~/.local/lib/python3.8/site-packages/poetry/packages/direct_origin.py:75 in get_package_from_directory
73│ @classmethod
74│ def get_package_from_directory(cls, directory: Path) -> Package:
→ 75│ return PackageInfo.from_directory(path=directory).to_package(root_dir=directory)
76│
77│ def get_package_from_url(self, url: str) -> Package:
1 ~/.local/lib/python3.8/site-packages/poetry/inspection/info.py:499 in from_directory
497│ info = cls.from_setup_files(path)
498│ else:
→ 499│ info = get_pep517_metadata(path)
500│ except PackageInfoError:
501│ if not info:
PackageInfoError
Unable to determine package info for path: /home/myfolder/poetry_env/poetry_env_demo/.venv/src/pytorch3d
Command ['/tmp/tmpxdbpwgq0/.venv/bin/python', '-'] errored with the following return code 1
Output:
Traceback (most recent call last):
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 466, in _handle_backend
yield
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 356, in get_requires_for_build
return set(get_requires(config_settings))
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
return self._call_hook('get_requires_for_build_wheel', {
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/pyproject_hooks/_impl.py", line 311, in _call_hook
self._subprocess_runner(
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 302, in _runner
self._hook_runner(cmd, cwd, extra_environ)
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/pyproject_hooks/_impl.py", line 71, in quiet_subprocess_runner
check_output(cmd, cwd=cwd, env=env, stderr=STDOUT)
File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/build-env-8vw3hcw7/bin/python', '/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmp9cn0rpj_']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 16, in <module>
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 356, in get_requires_for_build
return set(get_requires(config_settings))
File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/tmp/tmpxdbpwgq0/.venv/lib/python3.8/site-packages/build/__init__.py", line 474, in _handle_backend
raise BuildBackendException( # noqa: B904 # use raise from
build.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel
Input:
import build
import build.env
import pyproject_hooks
source = '/home/myfolder/poetry_env/poetry_env_demo/.venv/src/pytorch3d'
dest = '/tmp/tmpxdbpwgq0/dist'
with build.env.IsolatedEnvBuilder() as env:
builder = build.ProjectBuilder(
srcdir=source,
scripts_dir=env.scripts_dir,
python_executable=env.executable,
runner=pyproject_hooks.quiet_subprocess_runner,
)
env.install(builder.build_system_requires)
env.install(builder.get_requires_for_build('wheel'))
builder.metadata_path(dest)
Fallback egg_info generation failed.
Command ['/tmp/tmpxdbpwgq0/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1
Output:
Traceback (most recent call last):
File "setup.py", line 15, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
at ~/.local/lib/python3.8/site-packages/poetry/inspection/info.py:625 in get_pep517_metadata
621│ try:
622│ venv.run("python", "setup.py", "egg_info")
623│ info = PackageInfo.from_metadata(path)
624│ except EnvCommandError as fbe:
→ 625│ raise PackageInfoError(
626│ path, e, "Fallback egg_info generation failed.", fbe
627│ )
628│ finally:
629│ os.chdir(cwd)
Alright, so pytorch3d
has a dependency on torch
but despite this pre-requisite, the installation for some reason doesn't install the dependency. So I went ahead and installed torch
:
poetry add torch
Which went successfully. Now I retried the previous installation command, knowing that torch is certainly included in my environment by now:
poetry add -vvv git+https://github.com/facebookresearch/pytorch3d.git
But the error is exactly the same.
Edit 1
I saw this issue from earlier: #1619. But it doesn't offer a clear workaround on how to install pytorch3d with poetry. As I mentioned above, I also installed torch
separately before attempting to install pytorch3d
, but the installation still fails, as pytorch3d
somehow doesn't see the already present torch
dependency in the local venv.
Edit 2
I also came across this: #1479 . I assume this change would solve the issue, but it seems to have been never merged for some reason.