Rename top-level build folder to simplify wheel building

[jathu]

Based on the python docs, and the PyTorch wheel building workflow, building a wheel should just take the following steps:

# Installs deps referenced in pyproject.toml → build-system.requires
$ python3 -m pip install --upgrade build

# Builds the wheel
$ python3 -m build --wheel

Unfortunately, since we have a top-level folder called build, I think python is attempting to use that as a module — which is invalid. If we rename the folder from build, we should be able to simplify our CI setup to just those steps.

Although, the effort to rename the folder seems large and daunting. Is it worth the simplified workflow?

cc @larryliu0820 @lucylq

[GregoryComer]

I'm in favor of this. When I first started working on ET, I ended up doing cmake builds in the build directory without thinking multiple times and making a mess. Renaming our current build/ dir seems like a good idea.

[mergennachin]

The effort to rename the folder seems large and daunting. Is it worth the simplified workflow?

Why is this large and daunting? Isn't this a one-time codemod in fbsource?

[jathu]

Why is this large and daunting? Isn't this a one-time codemod in fbsource?

We have to capture all explicit usage of build/, but there might be also some implicit references. Not impossible, but would require a back and forth on updating, then letting the CI provide some feedback.

However, the bigger impact would be to update our docs that reference build scripts under that folder. This will also make any external tutorials stale since those scripts will not exist. Not a huge problem, but is it worth the effort to reduce wheel building by 1-2 steps?

[larryliu0820]

Installs deps referenced in pyproject.toml → build-system.requires

$ python3 -m pip install --upgrade build

Builds the wheel

$ python3 -m build --wheel

We might be able to ask pip to look into a different folder than build. We are currently using pip-out so maybe we can configure pip to look into pip-out?

[jathu]

We might be able to ask pip to look into a different folder than build. We are currently using pip-out so maybe we can configure pip to look into pip-out?

The problem is build is a module, and it's conflicting with our top-level folder name build when running python3 -m build --wheel (this essentially looks for ./build/__main__.py).

I tried referencing the module with relative and absolute paths, but it also throws an error:

$ python3 -m .venv/lib/python3.13/site-packages/build --wheel
/Users/jathu/executorch/.venv/bin/python3: Relative module names not supported

$ python3 -m /Users/jathu/executorch/.venv/lib/python3.13/site-packages/build --wheel
/Users/jathu/executorch/.venv/bin/python3: Error while finding module specification for '/Users/jathu/executorch/.venv/lib/python3.13/site-packages/build' (ModuleNotFoundError: No module named '/Users/jathu/executorch/')

[larryliu0820]

So this only happens when you run python3 -m build --wheel inside executorch/, right?

[larryliu0820]

Ah ok I just ran these commands locally and now I understand the issue. Yeah it seems there's no way around but rename build/.

[mergennachin]

However, the bigger impact would be to update our docs that reference build scripts under that folder. This will also make any external tutorials stale since those scripts will not exist. Not a huge problem, but is it worth the effort to reduce wheel building by 1-2 steps?

That's a general problem. But we have a way to solve today by version. We have two versions of the doc.

https://pytorch.org/executorch/main/ - that is mirror of the main branch
https://pytorch.org/executorch/0.5/ - that is mirror of the release branch

Depending on which versions of the code that they're, users will know which version of the doc to read.

[mergennachin]

We have to capture all explicit usage of build/, but there might be also some implicit references. Not impossible, but would require a back and forth on updating, then letting the CI provide some feedback.

Are there that many?

[mergennachin]

In terms of priority, it seems like a nice-to-have but not critical

Maybe a question for you instead, what does this enable moving forward

[larryliu0820]

I think on a high level it's preferred to not have a build/ in project root.

• Generally developers expect build/ to host temp files.
• Dev tools making assumptions that build/ contain build files:
  • vs code cmake plugin will automatically write CMake files into build/
  • github website doesn't index any filename inside build/

[mergennachin]

Okay, then let's do this rather early than later.