Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add documentation on how to do incremental builds #2796

Merged
merged 7 commits into from
Feb 7, 2024
Merged

Add documentation on how to do incremental builds #2796

Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e955203
Enable incremental build
pcmoritz Feb 7, 2024
0ad56b8
yapf
pcmoritz Feb 7, 2024
90dbcb5
remove hack
pcmoritz Feb 7, 2024
5d94f52
add documentation
pcmoritz Feb 7, 2024
476c120
add docs
pcmoritz Feb 7, 2024
ec1941f
update
pcmoritz Feb 7, 2024
2c33d74
format
pcmoritz Feb 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@
ROCM_SUPPORTED_ARCHS = {"gfx90a", "gfx942"}
# SUPPORTED_ARCHS = NVIDIA_SUPPORTED_ARCHS.union(ROCM_SUPPORTED_ARCHS)

if "VLLM_INCREMENTAL_BUILD_TORCH_PATH" in os.environ:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some document on how to use this? Setting the env var to

VLLM_INCREMENTAL_BUILD_TORCH_PATH=`python -c "import torch; print(torch.__path__[0])"`

is confusing to other users.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I added the docs, let me know what you think :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also tried to do VLLM_INCREMENTAL_BUILD=1 to make it easier btw, but I couldn't figure out how to get the global torch path from within setup.py -- I tried both via python utilities and using subprocess (both with and without shell), but I think there is some environment variable set which points the package dir to the temporary directory. It is probably best not to try to hack around this further :)

torch_cpp_ext._TORCH_PATH = os.environ["VLLM_INCREMENTAL_BUILD_TORCH_PATH"]


def _is_hip() -> bool:
return torch.version.hip is not None
Expand Down Expand Up @@ -432,6 +435,19 @@ def get_requirements() -> List[str]:
ext_modules = []
package_data["vllm"].append("*.so")

if "VLLM_INCREMENTAL_BUILD_TORCH_PATH" in os.environ:
# This is an optional hack to allow incremental compilation.
class VllmBuildExtension(BuildExtension):

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)

def build_extensions(self) -> None:
self.build_temp = "/tmp/vllmcompile/"
super().build_extensions()
else:
VllmBuildExtension = BuildExtension

setuptools.setup(
name="vllm",
version=get_vllm_version(),
Expand Down Expand Up @@ -459,6 +475,6 @@ def get_requirements() -> List[str]:
python_requires=">=3.8",
install_requires=get_requirements(),
ext_modules=ext_modules,
cmdclass={"build_ext": BuildExtension} if not _is_neuron() else {},
cmdclass={"build_ext": VllmBuildExtension} if not _is_neuron() else {},
package_data=package_data,
)
Loading