Skip to content

Commit 5afe90d

Browse files
authored
Disable tokenizers on Windows (#13667)
Tokenizers currently does not build on Windows. I've filed as meta-pytorch/tokenizers#111 and intend to fix this shortly, but I'm disabling for now to allow us to set up functional Windows CI and prevent regressions as we progressively enable things.
1 parent bda3142 commit 5afe90d

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

install_requirements.py

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -112,8 +112,13 @@ def install_requirements(use_pytorch_nightly):
112112

113113
LOCAL_REQUIREMENTS = [
114114
"third-party/ao", # We need the latest kernels for fast iteration, so not relying on pypi.
115-
"extension/llm/tokenizers", # TODO(larryliu0820): Setup a pypi package for this.
116-
]
115+
] + (
116+
[
117+
"extension/llm/tokenizers", # TODO(larryliu0820): Setup a pypi package for this.
118+
]
119+
if sys.platform != "win32"
120+
else []
121+
) # TODO(gjcomer): Re-enable when buildable on Windows.
117122

118123
# Install packages directly from local copy instead of pypi.
119124
# This is usually not recommended.

0 commit comments

Comments
 (0)