Skip to content

KV_cache offload #943

Open
Open
KV_cache offload#943

Description

@yuzhenmao
yuzhenmao
opened on Dec 15, 2024

Hi, I am using the latest huggingface transformers (version==4.48.0.dev0). When I tried to run the demo from here, I have this error: AttributeError: 'LlamaForCausalLM' object has no attribute 'set_kv_cache_offload'.

Does anyone know the solution to solve this issue? Thanks!

Activity

  1. kdw4537 commented on Mar 21, 2025

    @kdw4537
    kdw4537
    on Mar 21, 2025

    I'm having the same problem (OPT as well)
    Did you build a tokenizer in fork of transformers? In my case, failed.

  2. tjruwase commented on Mar 21, 2025

    @tjruwase
    tjruwase
    on Mar 21, 2025
    Contributor

    @yuzhenmao, sorry that we missed this issue. The kv cache offload was prototyped in a transformer's fork, but never upstreamted.
    https://github.com/deepspeedai/DeepSpeedExamples/blob/master/inference/huggingface/zero_inference/README.md#install

    @kdw4537, are you interested in this feature?

  3. Freed-Wu commented on Jun 12, 2025

    @Freed-Wu
    Freed-Wu
    on Jun 12, 2025

    It cannot be installed. @tjruwase

    Collecting git+https://github.com/tjruwase/transformers@kvcache-offload-cpu
  Cloning https://github.com/tjruwase/transformers (to revision kvcache-offload-cpu) to /tmp/pip-req-build-em4nbi06
  Running command git clone --filter=blob:none --quiet https://github.com/tjruwase/transformers /tmp/pip-req-build-em4nbi06
  Running command git checkout -b kvcache-offload-cpu --track origin/kvcache-offload-cpu
  Switched to a new branch 'kvcache-offload-cpu'
  branch 'kvcache-offload-cpu' set up to track 'origin/kvcache-offload-cpu'.
  Resolved https://github.com/tjruwase/transformers to commit 35515d7deb03200e0f835120ed40e817a9f6dd06
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: filelock in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (3.18.0)
Requirement already satisfied: huggingface-hub<1.0,>=0.15.1 in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (0.32.5)
Requirement already satisfied: numpy>=1.17 in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (2.3.0)
Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (25.0)
Requirement already satisfied: pyyaml>=5.1 in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (6.0.2)
Requirement already satisfied: regex!=2019.12.17 in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (2024.11.6)
Requirement already satisfied: requests in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (2.32.4)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.33.0.dev0)
  Using cached https://mirrors.bfsu.edu.cn/pypi/web/packages/29/9c/936ebad6dd963616189d6362f4c2c03a0314cf2a221ba15e48dd714d29cf/tokenizers-0.13.3.tar.gz (314 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: safetensors>=0.3.1 in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (0.5.3)
Requirement already satisfied: tqdm>=4.27 in /usr/lib/python3.13/site-packages (from transformers==4.33.0.dev0) (4.67.1)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/lib/python3.13/site-packages (from huggingface-hub<1.0,>=0.15.1->transformers==4.33.0.dev0) (2025.5.1)
Collecting hf-xet<2.0.0,>=1.1.2 (from huggingface-hub<1.0,>=0.15.1->transformers==4.33.0.dev0)
  Using cached https://mirrors.bfsu.edu.cn/pypi/web/packages/52/48/e929e6e3db6e4758c2adf0f2ca2c59287f1b76229d8bdc1a4c9cfc05212e/hf_xet-1.1.3-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/lib/python3.13/site-packages (from huggingface-hub<1.0,>=0.15.1->transformers==4.33.0.dev0) (4.13.2)
Requirement already satisfied: charset_normalizer<4,>=2 in /usr/lib/python3.13/site-packages (from requests->transformers==4.33.0.dev0) (3.4.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.13/site-packages (from requests->transformers==4.33.0.dev0) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3.13/site-packages (from requests->transformers==4.33.0.dev0) (2.4.0)
Building wheels for collected packages: transformers, tokenizers
  Building wheel for transformers (pyproject.toml) ... done
  Created wheel for transformers: filename=transformers-4.33.0.dev0-py3-none-any.whl size=7575341 sha256=eb695cf3411bccd53d24027237a4250556f9534fb8fd777fb4bb184d5f1563e1
  Stored in directory: /tmp/pip-ephem-wheel-cache-qykwx323/wheels/da/da/01/50a4b3b541a7de5557d73e3d11f3a432b6107eee12cb29cd1e
  Building wheel for tokenizers (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for tokenizers (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [659 lines of output]
      /tmp/pip-build-env-dizg4cry/overlay/lib/python3.13/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
      !!

              ********************************************************************************
              Please consider removing the following classifiers in favor of a SPDX license expression:

              License :: OSI Approved :: Apache Software License

              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************

      !!
        self._finalize_license_expression()
      running bdist_wheel
      running build
      running build_py
      creating build/lib.linux-x86_64-cpython-313/tokenizers
      copying py_src/tokenizers/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers
      creating build/lib.linux-x86_64-cpython-313/tokenizers/models
      copying py_src/tokenizers/models/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers/models
      creating build/lib.linux-x86_64-cpython-313/tokenizers/decoders
      copying py_src/tokenizers/decoders/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers/decoders
      creating build/lib.linux-x86_64-cpython-313/tokenizers/normalizers
      copying py_src/tokenizers/normalizers/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers/normalizers
      creating build/lib.linux-x86_64-cpython-313/tokenizers/pre_tokenizers
      copying py_src/tokenizers/pre_tokenizers/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers/pre_tokenizers
      creating build/lib.linux-x86_64-cpython-313/tokenizers/processors
      copying py_src/tokenizers/processors/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers/processors
      creating build/lib.linux-x86_64-cpython-313/tokenizers/trainers
      copying py_src/tokenizers/trainers/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers/trainers
      creating build/lib.linux-x86_64-cpython-313/tokenizers/implementations
      copying py_src/tokenizers/implementations/sentencepiece_unigram.py -> build/lib.linux-x86_64-cpython-313/tokenizers/implementations
      copying py_src/tokenizers/implementations/sentencepiece_bpe.py -> build/lib.linux-x86_64-cpython-313/tokenizers/implementations
      copying py_src/tokenizers/implementations/char_level_bpe.py -> build/lib.linux-x86_64-cpython-313/tokenizers/implementations
      copying py_src/tokenizers/implementations/byte_level_bpe.py -> build/lib.linux-x86_64-cpython-313/tokenizers/implementations
      copying py_src/tokenizers/implementations/bert_wordpiece.py -> build/lib.linux-x86_64-cpython-313/tokenizers/implementations
      copying py_src/tokenizers/implementations/base_tokenizer.py -> build/lib.linux-x86_64-cpython-313/tokenizers/implementations
      copying py_src/tokenizers/implementations/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers/implementations
      creating build/lib.linux-x86_64-cpython-313/tokenizers/tools
      copying py_src/tokenizers/tools/visualizer.py -> build/lib.linux-x86_64-cpython-313/tokenizers/tools
      copying py_src/tokenizers/tools/__init__.py -> build/lib.linux-x86_64-cpython-313/tokenizers/tools
      copying py_src/tokenizers/__init__.pyi -> build/lib.linux-x86_64-cpython-313/tokenizers
      copying py_src/tokenizers/models/__init__.pyi -> build/lib.linux-x86_64-cpython-313/tokenizers/models
      copying py_src/tokenizers/decoders/__init__.pyi -> build/lib.linux-x86_64-cpython-313/tokenizers/decoders
      copying py_src/tokenizers/normalizers/__init__.pyi -> build/lib.linux-x86_64-cpython-313/tokenizers/normalizers
      copying py_src/tokenizers/pre_tokenizers/__init__.pyi -> build/lib.linux-x86_64-cpython-313/tokenizers/pre_tokenizers
      copying py_src/tokenizers/processors/__init__.pyi -> build/lib.linux-x86_64-cpython-313/tokenizers/processors
      copying py_src/tokenizers/trainers/__init__.pyi -> build/lib.linux-x86_64-cpython-313/tokenizers/trainers
      copying py_src/tokenizers/tools/visualizer-styles.css -> build/lib.linux-x86_64-cpython-313/tokenizers/tools
      running build_ext
      running build_rust
          Updating `bfsu` index
           Locking 300 packages to latest compatible versions
            Adding env_logger v0.7.1 (available: v0.11.8)
            Adding itertools v0.9.0 (available: v0.14.0)
            Adding ndarray v0.13.1 (available: v0.16.1)
            Adding numpy v0.18.0 (available: v0.25.0)
            Adding pyo3 v0.18.3 (available: v0.25.0)
      cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --
         Compiling libc v0.2.172
         Compiling proc-macro2 v1.0.95
         Compiling unicode-ident v1.0.18
         Compiling autocfg v1.4.0
         Compiling shlex v1.3.0
         Compiling cfg-if v1.0.1
         Compiling pkg-config v0.3.32
         Compiling memchr v2.7.5
         Compiling stable_deref_trait v1.2.0
         Compiling once_cell v1.21.3
         Compiling version_check v0.9.5
         Compiling typenum v1.18.0
         Compiling syn v1.0.109
         Compiling pin-project-lite v0.2.16
         Compiling bitflags v2.9.1
         Compiling smallvec v1.15.1
         ...
         Compiling tokenizers v0.13.3 (/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/tokenizers-lib)
           Running `rustc --crate-name tokenizers --edition=2018 tokenizers-lib/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="cached-path"' --cfg 'feature="clap"' --cfg 'feature="cli"' --cfg 'feature="default"' --cfg 'feature="dirs"' --cfg 'feature="esaxx_fast"' --cfg 'feature="http"' --cfg 'feature="indicatif"' --cfg 'feature="onig"' --cfg 'feature="progressbar"' --cfg 'feature="reqwest"' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values("cached-path", "clap", "cli", "default", "dirs", "esaxx_fast", "fancy-regex", "http", "indicatif", "onig", "progressbar", "reqwest", "unstable_wasm"))' -C metadata=86ea2528eed6c520 -C extra-filename=-63e70014eb414976 --out-dir /tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps -C strip=debuginfo -L dependency=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps --extern aho_corasick=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libaho_corasick-8be2932ba1d4f103.rmeta --extern cached_path=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libcached_path-7e016c0836af2b36.rmeta --extern clap=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libclap-fa9a37346145174a.rmeta --extern derive_builder=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libderive_builder-a43212fd1b428fb6.rmeta --extern dirs=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libdirs-7cd6ed3e2648e17a.rmeta --extern esaxx_rs=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libesaxx_rs-06374b8edc4393f4.rmeta --extern getrandom=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libgetrandom-c78eb40ece4a1da6.rmeta --extern indicatif=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libindicatif-b32862cb28ea595a.rmeta --extern itertools=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libitertools-5a65b501f315af7a.rmeta --extern lazy_static=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/liblazy_static-95b98c7937dd5b53.rmeta --extern log=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/liblog-c930a65bd2d33389.rmeta --extern macro_rules_attribute=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libmacro_rules_attribute-8ccf184cad68b546.rmeta --extern monostate=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libmonostate-9ecf66afef857f1e.rmeta --extern onig=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libonig-4c8708f0dcae69c7.rmeta --extern paste=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libpaste-25f1f0e59fe3fce4.so --extern rand=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/librand-585aa8890a2a7095.rmeta --extern rayon=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/librayon-664a725f0e3fb01b.rmeta --extern rayon_cond=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/librayon_cond-aaa74168f63f18ac.rmeta --extern regex=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libregex-3df5e2487fe5240f.rmeta --extern regex_syntax=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libregex_syntax-8cfad6fd4030a27d.rmeta --extern reqwest=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libreqwest-8113c37e66f04d9c.rmeta --extern serde=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libserde-3304991411b95fb4.rmeta --extern serde_json=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libserde_json-5e260cc830bb35cb.rmeta --extern spm_precompiled=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libspm_precompiled-4808fa2d00f97681.rmeta --extern thiserror=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libthiserror-b855120315a6a169.rmeta --extern unicode_normalization_alignments=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libunicode_normalization_alignments-9135a16f27f13bc2.rmeta --extern unicode_segmentation=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libunicode_segmentation-2a9ca491bcb513eb.rmeta --extern unicode_categories=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libunicode_categories-5f7a76af5f0a221c.rmeta -L native=/usr/lib -L native=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/build/zstd-sys-fd2eb66a5411554c/out -L native=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/build/esaxx-rs-6af3e3eb6afb5ee1/out -L native=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/build/onig_sys-33dbd85663e96a6e/out`
      warning: variable does not need to be mutable
         --> tokenizers-lib/src/models/unigram/model.rs:265:21
          |
      265 |                 let mut target_node = &mut best_path_ends_at[key_pos];
          |                     ----^^^^^^^^^^^
          |                     |
          |                     help: remove this `mut`
          |
          = note: `#[warn(unused_mut)]` on by default

      warning: variable does not need to be mutable
         --> tokenizers-lib/src/models/unigram/model.rs:282:21
          |
      282 |                 let mut target_node = &mut best_path_ends_at[starts_at + mblen];
          |                     ----^^^^^^^^^^^
          |                     |
          |                     help: remove this `mut`

      warning: variable does not need to be mutable
         --> tokenizers-lib/src/pre_tokenizers/byte_level.rs:200:59
          |
      200 |     encoding.process_tokens_with_offsets_mut(|(i, (token, mut offsets))| {
          |                                                           ----^^^^^^^
          |                                                           |
          |                                                           help: remove this `mut`

      error: casting `&T` to `&mut T` is undefined behavior, even if the reference is unused, consider instead using an `UnsafeCell`
         --> tokenizers-lib/src/models/bpe/trainer.rs:526:47
          |
      522 |                     let w = &words[*i] as *const _ as *mut _;
          |                             -------------------------------- casting happened here
      ...
      526 |                         let word: &mut Word = &mut (*w);
          |                                               ^^^^^^^^^
          |
          = note: for more information, visit <https://doc.rust-lang.org/book/ch15-05-interior-mutability.html>
          = note: `#[deny(invalid_reference_casting)]` on by default

      warning: `tokenizers` (lib) generated 3 warnings
      error: could not compile `tokenizers` (lib) due to 1 previous error; 3 warnings emitted

      Caused by:
        process didn't exit successfully: `rustc --crate-name tokenizers --edition=2018 tokenizers-lib/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="cached-path"' --cfg 'feature="clap"' --cfg 'feature="cli"' --cfg 'feature="default"' --cfg 'feature="dirs"' --cfg 'feature="esaxx_fast"' --cfg 'feature="http"' --cfg 'feature="indicatif"' --cfg 'feature="onig"' --cfg 'feature="progressbar"' --cfg 'feature="reqwest"' --check-cfg 'cfg(docsrs,test)' --check-cfg 'cfg(feature, values("cached-path", "clap", "cli", "default", "dirs", "esaxx_fast", "fancy-regex", "http", "indicatif", "onig", "progressbar", "reqwest", "unstable_wasm"))' -C metadata=86ea2528eed6c520 -C extra-filename=-63e70014eb414976 --out-dir /tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps -C strip=debuginfo -L dependency=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps --extern aho_corasick=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libaho_corasick-8be2932ba1d4f103.rmeta --extern cached_path=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libcached_path-7e016c0836af2b36.rmeta --extern clap=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libclap-fa9a37346145174a.rmeta --extern derive_builder=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libderive_builder-a43212fd1b428fb6.rmeta --extern dirs=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libdirs-7cd6ed3e2648e17a.rmeta --extern esaxx_rs=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libesaxx_rs-06374b8edc4393f4.rmeta --extern getrandom=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libgetrandom-c78eb40ece4a1da6.rmeta --extern indicatif=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libindicatif-b32862cb28ea595a.rmeta --extern itertools=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libitertools-5a65b501f315af7a.rmeta --extern lazy_static=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/liblazy_static-95b98c7937dd5b53.rmeta --extern log=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/liblog-c930a65bd2d33389.rmeta --extern macro_rules_attribute=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libmacro_rules_attribute-8ccf184cad68b546.rmeta --extern monostate=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libmonostate-9ecf66afef857f1e.rmeta --extern onig=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libonig-4c8708f0dcae69c7.rmeta --extern paste=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libpaste-25f1f0e59fe3fce4.so --extern rand=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/librand-585aa8890a2a7095.rmeta --extern rayon=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/librayon-664a725f0e3fb01b.rmeta --extern rayon_cond=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/librayon_cond-aaa74168f63f18ac.rmeta --extern regex=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libregex-3df5e2487fe5240f.rmeta --extern regex_syntax=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libregex_syntax-8cfad6fd4030a27d.rmeta --extern reqwest=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libreqwest-8113c37e66f04d9c.rmeta --extern serde=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libserde-3304991411b95fb4.rmeta --extern serde_json=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libserde_json-5e260cc830bb35cb.rmeta --extern spm_precompiled=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libspm_precompiled-4808fa2d00f97681.rmeta --extern thiserror=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libthiserror-b855120315a6a169.rmeta --extern unicode_normalization_alignments=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libunicode_normalization_alignments-9135a16f27f13bc2.rmeta --extern unicode_segmentation=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libunicode_segmentation-2a9ca491bcb513eb.rmeta --extern unicode_categories=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/deps/libunicode_categories-5f7a76af5f0a221c.rmeta -L native=/usr/lib -L native=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/build/zstd-sys-fd2eb66a5411554c/out -L native=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/build/esaxx-rs-6af3e3eb6afb5ee1/out -L native=/tmp/pip-install-bk5z726k/tokenizers_7a7b16d37ce74c718e6bf722333a98db/target/release/build/onig_sys-33dbd85663e96a6e/out` (exit status: 1)
      error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --` failed with code 101
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Successfully built transformers
Failed to build tokenizers
ERROR: Failed to build installable wheels for some pyproject.toml based projects (tokenizers)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions