Skip to content

A short term fix for editable mode install failed to import root level module such as exir #9818

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 2, 2025

Conversation

larryliu0820
Copy link
Contributor

@larryliu0820 larryliu0820 commented Apr 1, 2025

Summary:

Fixes #9558.

The src/executorch/exir file exists primarily due to the limitations of pip install in editable mode. Specifically, pip install -e . does not recognize <executorch root>/exir (or any root level directory with a __init__.py) as a valid package module because of the presence of <executorch root>/exir/__init__.py. See the following GitHub issue for details: Issue #9558.

To work around this limitation, a symlink is used. With this symlink and this package entry in pyproject.toml:

[tool.setuptools.package-dir]
# ...
"executorch" = "src/executorch"

We are telling pip install -e . to treat src/executorch as the root of the executorch package and hence mapping executorch.exir to src/executorch/exir. This effectively gets exir out from the root level package.

This allows us to perform pip install -e . successfully and enables the execution of the following command:

python -c "from executorch.exir import CaptureConfig"

Test Plan:

./install_executorch.sh --pybind --editable
python -c "from executorch.exir import CaptureConfig"

Reviewers:

Subscribers:

Tasks:

Tags:

…l module such as exir

Summary:

Fixes #9558.

The `src/executorch/exir` file exists primarily due to the limitations of `pip install` in editable mode. Specifically, `pip install -e .` does not recognize `<executorch root>/exir` (or any root level directory with a `__init__.py`) as a valid package module because of the presence of `<executorch root>/exir/__init__.py`. See the following GitHub issue for details: [Issue #9558](#9558).

To work around this limitation, a symlink is used. With this symlink and this package entry in `pyproject.toml`:

```toml
[tool.setuptools.package-dir]
...
"executorch" = "src/executorch"
```
We are telling `pip install -e .` to treat `src/executorch` as the root of the `executorch` package and hence mapping `executorch.exir` to `src/executorch/exir`.

This allows us to perform `pip install -e .` successfully and enables the execution of the following command:

```bash
python -c "from executorch.exir import CaptureConfig"
```

Test Plan:

```bash
./install_executorch.sh --pybind --editable
python -c "from executorch.exir import CaptureConfig"
```

Reviewers:

Subscribers:

Tasks:

Tags:
Copy link

pytorch-bot bot commented Apr 1, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9818

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 3 Pending, 1 Unrelated Failure

As of commit f599c9f with merge base b66c319 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 1, 2025
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@larryliu0820 larryliu0820 changed the title A short term fix for editable mode install failed to import root leve… A short term fix for editable mode install failed to import root level module such as exir Apr 2, 2025
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@mergennachin
Copy link
Contributor

Why only exir?

What if we have a symlink from root/src/executorch to root/ as opposed to only focusing on root/src/executorch/exir?

@larryliu0820
Copy link
Contributor Author

larryliu0820 commented Apr 2, 2025

Why only exir?

What if we have a symlink from root/src/executorch to root/ as opposed to only focusing on root/src/executorch/exir?

Gee, I think that's better. I don't think we should symlink the whole root directory, instead we probably want to symlink all next level directories with python file.

@larryliu0820 larryliu0820 added release notes: build Changes related to build, including dependency upgrades, build flags, optimizations, etc. and removed topic: not user facing labels Apr 2, 2025
Copy link
Contributor

@swolchok swolchok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, is exir really the only top-level directory where this is a problem? I'm surprised, but it's not impossible, so here's an accept.

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@larryliu0820
Copy link
Contributor Author

Wow, is exir really the only top-level directory where this is a problem? I'm surprised, but it's not impossible, so here's an accept.

Actually tested

python -c "from executorch.examples.models import MODEL_NAME_TO_MODEL"

Same problem so in this update I symlink'd all the modules

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@swolchok
Copy link
Contributor

swolchok commented Apr 2, 2025

@mergennachin you probably want to take a look at this

@larryliu0820 larryliu0820 merged commit 1e64fa2 into main Apr 2, 2025
172 of 173 checks passed
@larryliu0820 larryliu0820 deleted the fix_import branch April 2, 2025 22:10
kirklandsign pushed a commit that referenced this pull request Apr 11, 2025
…l module such as exir (#9818)

Summary:

Fixes #9558.

The `src/executorch/exir` file exists primarily due to the limitations
of `pip install` in editable mode. Specifically, `pip install -e .` does
not recognize `<executorch root>/exir` (or any root level directory with
a `__init__.py`) as a valid package module because of the presence of
`<executorch root>/exir/__init__.py`. See the following GitHub issue for
details: [Issue
#9558](#9558).

To work around this limitation, a symlink is used. With this symlink and
this package entry in `pyproject.toml`:

```toml
[tool.setuptools.package-dir]
# ...
"executorch" = "src/executorch"
```
We are telling `pip install -e .` to treat `src/executorch` as the root
of the `executorch` package and hence mapping `executorch.exir` to
`src/executorch/exir`. This effectively gets `exir` out from the root
level package.

This allows us to perform `pip install -e .` successfully and enables
the execution of the following command:

```bash
python -c "from executorch.exir import CaptureConfig"
```

Test Plan:

```bash
./install_executorch.sh --pybind --editable
python -c "from executorch.exir import CaptureConfig"
```

Reviewers:

Subscribers:

Tasks:

Tags:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: build Changes related to build, including dependency upgrades, build flags, optimizations, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

from executorch.exir import CaptureConfig fails with ExecuTorch pip installed in editable mode
4 participants