build error

[geraldstanje1]

hi @dmitry-tokarev-nv @mc-nv @achartier @chzblych

can someone tell me why i see the following error?
it should build triton inference server with tensorrt llm backend?
also why is it using triton server with 25.06 as a base image (https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/dockerfile/Dockerfile.triton.trt_llm_backend#L1-L2) - how would that work when using 25.09? - will it come with llm api 1.0?

how can i build the smallest possible image with triton inference server 25.09, tensorrt llm and llm api 1.0

git clone https://github.com/triton-inference-server/tensorrtllm_backend
Cloning into 'tensorrtllm_backend'...
remote: Enumerating objects: 3199, done.        
remote: Counting objects: 100% (1016/1016), done.        
remote: Compressing objects: 100% (375/375), done.        
remote: Total 3199 (delta 848), reused 649 (delta 641), pack-reused 2183 (from 4)        
Receiving objects: 100% (3199/3199), 1.50 MiB | 25.62 MiB/s, done.
Resolving deltas: 100% (1806/1806), done.

⚡ ~ cd tensorrtllm_backend
⚡ main ~/tensorrtllm_backend git lfs install && git submodule update --init --recursive
Updated Git hooks.
Git LFS initialized.
Submodule 'tensorrt_llm' (https://github.com/NVIDIA/TensorRT-LLM.git) registered for path 'tensorrt_llm'
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm'...
Submodule path 'tensorrt_llm': checked out '560ded5450b79efde720162fc397d7efa59aae6d'
Submodule '3rdparty/DeepGEMM' (https://github.com/ruoqianguo/DeepGEMM.git) registered for path 'tensorrt_llm/3rdparty/DeepGEMM'
Submodule '3rdparty/NVTX' (https://github.com/NVIDIA/NVTX.git) registered for path 'tensorrt_llm/3rdparty/NVTX'
Submodule '3rdparty/cppzmq' (https://github.com/zeromq/cppzmq.git) registered for path 'tensorrt_llm/3rdparty/cppzmq'
Submodule '3rdparty/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'tensorrt_llm/3rdparty/cutlass'
Submodule '3rdparty/cxxopts' (https://github.com/jarro2783/cxxopts) registered for path 'tensorrt_llm/3rdparty/cxxopts'
Submodule '3rdparty/json' (https://github.com/nlohmann/json.git) registered for path 'tensorrt_llm/3rdparty/json'
Submodule '3rdparty/nanobind' (https://github.com/wjakob/nanobind) registered for path 'tensorrt_llm/3rdparty/nanobind'
Submodule '3rdparty/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'tensorrt_llm/3rdparty/pybind11'
Submodule '3rdparty/ucxx' (https://github.com/rapidsai/ucxx.git) registered for path 'tensorrt_llm/3rdparty/ucxx'
Submodule '3rdparty/xgrammar' (https://github.com/mlc-ai/xgrammar.git) registered for path 'tensorrt_llm/3rdparty/xgrammar'
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/DeepGEMM'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/NVTX'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/cppzmq'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/cutlass'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/cxxopts'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/json'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/nanobind'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/pybind11'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/ucxx'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/xgrammar'...
Submodule path 'tensorrt_llm/3rdparty/DeepGEMM': checked out '0315934ce27c5c0b05dfff1a5eb101a5f8872cfe'
Submodule 'third-party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'tensorrt_llm/3rdparty/DeepGEMM/third-party/cutlass'
Submodule 'third-party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'tensorrt_llm/3rdparty/DeepGEMM/third-party/fmt'
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/DeepGEMM/third-party/cutlass'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/DeepGEMM/third-party/fmt'...
Submodule path 'tensorrt_llm/3rdparty/DeepGEMM/third-party/cutlass': checked out 'a49a78ffefc86a87160dfe0ccc3a3a2d1622c918'
Submodule path 'tensorrt_llm/3rdparty/DeepGEMM/third-party/fmt': checked out '553ec11ec06fbe0beebfbb45f9dc3c9eabd83d28'
Submodule path 'tensorrt_llm/3rdparty/NVTX': checked out 'a1ceb0677f67371ed29a2b1c022794f077db5fe7'
Submodule path 'tensorrt_llm/3rdparty/cppzmq': checked out 'c94c20743ed7d4aa37835a5c46567ab0790d4acc'
Submodule path 'tensorrt_llm/3rdparty/cutlass': checked out '57e3cfb47a2d9e0d46eb6335c3dc411498efa198'
Submodule path 'tensorrt_llm/3rdparty/cxxopts': checked out 'eb787304d67ec22f7c3a184ee8b4c481d04357fd'
Submodule path 'tensorrt_llm/3rdparty/json': checked out '55f93686c01528224f448c19128836e7df245f72'
Submodule path 'tensorrt_llm/3rdparty/nanobind': checked out 'a0ed2587f1089ef7657e2ed49ad6756b01c74e9f'
Submodule 'ext/robin_map' (https://github.com/Tessil/robin-map) registered for path 'tensorrt_llm/3rdparty/nanobind/ext/robin_map'
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/nanobind/ext/robin_map'...
Submodule path 'tensorrt_llm/3rdparty/nanobind/ext/robin_map': checked out '4ec1bf19c6a96125ea22062f38c2cf5b958e448e'
Submodule path 'tensorrt_llm/3rdparty/pybind11': checked out 'f99ffd7e03001810a3e722bf48ad1a9e08415d7d'
Submodule path 'tensorrt_llm/3rdparty/ucxx': checked out '16eaa57c8d98c8ef54d666a2d2b11e76cfa565f5'
Submodule path 'tensorrt_llm/3rdparty/xgrammar': checked out 'e4e816f5f0fe39f5b1601a17a4552307fa3b70ff'
Submodule '3rdparty/cpptrace' (https://github.com/jeremy-rifkin/cpptrace.git) registered for path 'tensorrt_llm/3rdparty/xgrammar/3rdparty/cpptrace'
Submodule '3rdparty/dlpack' (https://github.com/dmlc/dlpack.git) registered for path 'tensorrt_llm/3rdparty/xgrammar/3rdparty/dlpack'
Submodule '3rdparty/googletest' (https://github.com/google/googletest.git) registered for path 'tensorrt_llm/3rdparty/xgrammar/3rdparty/googletest'
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/xgrammar/3rdparty/cpptrace'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/xgrammar/3rdparty/dlpack'...
Cloning into '/teamspace/studios/this_studio/tensorrtllm_backend/tensorrt_llm/3rdparty/xgrammar/3rdparty/googletest'...
Submodule path 'tensorrt_llm/3rdparty/xgrammar/3rdparty/cpptrace': checked out '6689d14c203eed390ae7bb64f56a983cfd7dff9c'
Submodule path 'tensorrt_llm/3rdparty/xgrammar/3rdparty/dlpack': checked out 'bbd2f4d32427e548797929af08cfe2a9cbb3cf12'
Submodule path 'tensorrt_llm/3rdparty/xgrammar/3rdparty/googletest': checked out 'df1544bcee0c7ce35cd5ea0b3eb8cc81855a4140'
⚡ main ~/tensorrtllm_backend DOCKER_BUILDKIT=1 docker build -t triton_trt_llm -f dockerfile/Dockerfile.triton.trt_llm_backend .
[+] Building 282.6s (6/49)                                                                                                                                             docker:default
 => [internal] load build definition from Dockerfile.triton.trt_llm_backend                                                                                                      0.0s
 => => transferring dockerfile: 10.02kB                                                                                                                                          0.0s
 => [internal] load metadata for nvcr.io/nvidia/tritonserver:25.06-py3-min                                                                                                       0.6s
 => [internal] load metadata for nvcr.io/nvidia/pytorch:25.06-py3                                                                                                                0.6s
 => [internal] load .dockerignore                                                                                                                                                0.0s
 => => transferring context: 2B                                                                                                                                                  0.0s
 => CANCELED [pytorch_image 1/1] FROM nvcr.io/nvidia/pytorch:25.06-py3@sha256:025d9b102b5436d4af8af58f12c6a46b7e5d16f19543b1d2cc4446bf2650b4f1                                 281.9s
 => => resolve nvcr.io/nvidia/pytorch:25.06-py3@sha256:025d9b102b5436d4af8af58f12c6a46b7e5d16f19543b1d2cc4446bf2650b4f1                                                          0.0s
 => => sha256:025d9b102b5436d4af8af58f12c6a46b7e5d16f19543b1d2cc4446bf2650b4f1 745B / 745B                                                                                       0.0s
 => => sha256:06aa7e7a6f5a5633ff3b34fb4f016a66d3a3f85c108e13270c2fba3f7fe6a93c 52.69kB / 52.69kB                                                                                 0.0s
 => => sha256:cebaad6571a97399701575be85f05a33d3e7b5a0f24fc5473510a51e2aa673eb 3.83MB / 3.83MB                                                                                   0.1s
 => => sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B / 32B                                                                                         0.1s
 => => sha256:3cb18e2c438db8af2d3a659ca27fac5da328640261c38c48a34edcd223c38af9 12.73kB / 12.73kB                                                                                 0.0s
 => => sha256:1f7fdc9ac51458ab136ae88743d5bbfc0d2b57717f872349ff9ada895b600aea 16.91kB / 16.91kB                                                                                 0.3s
 => => sha256:3c313eb917ea07f6a2e8e8278cf840357ac5b88b3464f471efa90cedadc1e65a 114.38MB / 114.38MB                                                                               0.7s
 => => sha256:7848405aa76a398a6014ea511d9b0b52fb00dafa8e3e24ce3e0d85982fdaff16 89.88MB / 89.88MB                                                                                 0.5s
 => => sha256:d1c1b0ca7e576b38870258a7c0667ad16b820970c171ee36dc50ba041dc0f1dd 11.53kB / 11.53kB                                                                                 0.3s
 => => sha256:9f11a52127b5ce6c65cf99d5afa9fb7f5b0fe56d3f34c2b3ab4670e08259564b 179B / 179B                                                                                       0.4s
 => => sha256:1d7d94b9f382ff484bf5c356f41fae712f2fa9a777d5f0ffe52f6e90833372e0 1.73MB / 1.73MB                                                                                   0.5s
 => => sha256:f1af243d1e830ae7d4d0c27b2a39f13ebbd20eb3a64fc9ded54aada16af15d56 5.01kB / 5.01kB                                                                                   0.5s
 => => sha256:fa8ec00ab1d43016e10ba67c230b6a2c742b4c93ba344147ea75250ffe465db9 407B / 407B                                                                                       0.6s
 => => sha256:3f893d6c81004ca97381c46df634bc2bbc054b5b63e9831a8fa85984b51ca89d 6.37kB / 6.37kB                                                                                   0.5s
 => => sha256:e21e46372e43c0c8ddd5593327607f990ed6c5ef7d9a81f4423e7b7a001edd69 216.26MB / 216.26MB                                                                               1.6s
 => => sha256:47d8c16719752ae1120048af33336b3f82a7b613898c85e6f3981ae8d640cad2 106.39MB / 106.39MB                                                                               1.1s
 => => sha256:d260da74816ae1ec63adc9f90044c2a2e96f143450997fecc0a98eafc3bc0de5 4.71GB / 5.13GB                                                                                 282.0s
 => => sha256:e327fded2fe59c3698beb0943cf7845ab5d7d5dba445e1d04b13aaba7424f0c6 35.96MB / 35.96MB                                                                                 1.3s
 => => sha256:98d1cd1e264fb1e189debdfbcb78ae9aab91407e17b0a4d3515e347e0b8669df 503B / 503B                                                                                       1.3s
 => => sha256:57e194be082f9502172275266c867e10163bda932800b03f422d54504dd9833e 354.89MB / 354.89MB                                                                               3.2s
 => => sha256:d9d352c11bbd3880007953ed6eec1cbace76898828f3434984a0ca60672fdf5a 29.72MB / 29.72MB                                                                                 1.9s
 => => extracting sha256:d9d352c11bbd3880007953ed6eec1cbace76898828f3434984a0ca60672fdf5a                                                                                        0.9s
 => => extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1                                                                                        0.0s
 => => extracting sha256:cebaad6571a97399701575be85f05a33d3e7b5a0f24fc5473510a51e2aa673eb                                                                                        0.1s
 => => sha256:2fe22e30ede5741ed908721d70dc83fae5b64f149b00b786aaefbe0d059f47f2 73.08MB / 73.08MB                                                                                 3.6s
 => => extracting sha256:7848405aa76a398a6014ea511d9b0b52fb00dafa8e3e24ce3e0d85982fdaff16                                                                                        1.6s
 => => sha256:c58367625a61af8b6ebbcc01de857bd7e4d854139e710f67ef5d7976ca591af1 5.71MB / 5.71MB                                                                                   3.6s
 => => sha256:1fcc2095320523937f930517a5f6f0859d931565a4e9a0ebd3d7d1c0f5a855dd 2.80kB / 2.80kB                                                                                   3.7s
 => => sha256:c1118e55ca0a0d13fd9068a67605b74646790e581efc4c54a33d5607d51aac6f 29.81MB / 29.81MB                                                                                 3.9s
 => => sha256:1fd0643fa149dd996c1f0a366bfd6720b8fad0e0364a163768e97ba408ace388 116B / 116B                                                                                       4.0s
 => => sha256:b632ad189d81063b5d6052dc22bbe4b836037c26f4456ce8bad6afd17844965b 407.45MB / 407.45MB                                                                               6.2s
 => => extracting sha256:3c313eb917ea07f6a2e8e8278cf840357ac5b88b3464f471efa90cedadc1e65a                                                                                        1.4s
 => => sha256:ccf993b4001ed5d341713fe63ba48b279b88da540df833038948e63bfb3d8370 671.94MB / 671.94MB                                                                               9.7s
 => => extracting sha256:1f7fdc9ac51458ab136ae88743d5bbfc0d2b57717f872349ff9ada895b600aea                                                                                        0.0s
 => => extracting sha256:d1c1b0ca7e576b38870258a7c0667ad16b820970c171ee36dc50ba041dc0f1dd                                                                                        0.0s
 => => extracting sha256:9f11a52127b5ce6c65cf99d5afa9fb7f5b0fe56d3f34c2b3ab4670e08259564b                                                                                        0.0s
 => => extracting sha256:1d7d94b9f382ff484bf5c356f41fae712f2fa9a777d5f0ffe52f6e90833372e0                                                                                        0.0s
 => => extracting sha256:f1af243d1e830ae7d4d0c27b2a39f13ebbd20eb3a64fc9ded54aada16af15d56                                                                                        0.0s
 => => extracting sha256:3f893d6c81004ca97381c46df634bc2bbc054b5b63e9831a8fa85984b51ca89d                                                                                        0.0s
 => => extracting sha256:fa8ec00ab1d43016e10ba67c230b6a2c742b4c93ba344147ea75250ffe465db9                                                                                        0.0s
 => => extracting sha256:47d8c16719752ae1120048af33336b3f82a7b613898c85e6f3981ae8d640cad2                                                                                        2.0s
 => => extracting sha256:e21e46372e43c0c8ddd5593327607f990ed6c5ef7d9a81f4423e7b7a001edd69                                                                                        2.4s
 => => sha256:d54123f469003400f16a86b6f8576e6804e699ff943ac894a5f93bb00e7b1e67 75.81MB / 75.81MB                                                                                10.3s
 => => sha256:dbf7276a508b9d89fd431c48f823dd6f201c60c376d7eb8edf6a6b8e4f481e5e 8.56kB / 8.56kB                                                                                  10.4s
 => => sha256:b5e287539ab025b44df56d5b4641e7d339c74e48717159c7cec24e4e2a2020f1 4.88MB / 4.88MB                                                                                  10.7s
 => => sha256:8fc3df16a73cd10c00b3bbc7c595cba492677f4b6da773d000d6ea314ab33cd9 521B / 521B                                                                                      10.7s
 => => sha256:7d731a472eaf55482ccb545d31d7cbfea04325316a2686c800f7c1a021a09458 364B / 364B                                                                                      10.8s
 => => sha256:a3dffce50035c2ac1a5ae5a8f1b3fd879db4245f97671623d5519e3ccfa9eec3 377B / 377B                                                                                      10.8s
 => => sha256:ea3b419012474d08968b9424ebe59a02cba858dbb4a72d763e17d7dc28b9dc63 1.27GB / 1.27GB                                                                                  18.2s
 => => sha256:95bfc6ee1340d9a449ff244e161df747489f98440f77d9afd67b208ddb5ab3a7 2.97kB / 2.97kB                                                                                  18.2s
 => => sha256:1ff567a80e8a46262e645ac5af566f00155e3bc4cfabe168a8195d4a37d1591e 2.72MB / 2.72MB                                                                                  18.3s
 => => sha256:ef7ac0d6c440c31b722ded7c1c01e66d548cd6e123c8127f18caec6bf2c3176c 71.92kB / 71.92kB                                                                                18.3s
 => => sha256:833ceef9466d5e6d0dc4c159274c582ee84a4a9f9131210f7422d2033df93dc5 2.87MB / 2.87MB                                                                                  18.4s
 => => sha256:b834afe576443d7566979f3ffcb248cccfab74fe1254bb0cb1e05c3fbcd89dfa 6.10MB / 6.10MB                                                                                  18.5s
 => => sha256:f99b645a1a7f59afb1cd425645ce5cc9aa6513244c9a02575a6aeca42c617a7d 264.31MB / 264.31MB                                                                              19.9s
 => => sha256:f6d715971816912d8a04b3381ee0a8e118175093e588cbc09347a5e4e4e3243c 529.11MB / 529.11MB                                                                              22.6s
 => => sha256:1efbbb3e559e09f3a20bb0f88be259cc9280605fd649cb00f83d7b53e57690f6 23.12MB / 23.12MB                                                                                22.9s
 => => sha256:9e7a7a2f250f43a9174d5a660be035819ff524544490368408d0b284345a879b 1.11kB / 1.11kB                                                                                  23.1s
 => => sha256:ba009bd75d43780e3d6125a7e5051289bcef186c18661bcbee613ca23d0c0da2 83.63MB / 83.63MB                                                                                24.0s
 => => sha256:5d95a8fe9ba514a81cd01f95081deb262a887fef0d21d51d631cf9f34031d399 73.67kB / 73.67kB                                                                                24.2s
 => => sha256:3fd2c195086d0156b89794ac02fb914a65f1c02006705985683f921903c18e07 1.68GB / 1.68GB                                                                                  37.1s
 => => sha256:a2bea83f54add0bef395ceff905f98b55c4f7ad6acf1ac5fa4604b4fc5548c17 1.12kB / 1.12kB                                                                                  37.1s
 => => sha256:cafe055054516295bff7d421f3042e33edc298080c59559969c2fab17141c583 1.11kB / 1.11kB                                                                                  37.2s
 => => sha256:9963b765221fefe8fac82ebfb5ac0a08d4d60624c6593eb4e24d7185339ab39c 30.13MB / 30.13MB                                                                                37.5s
 => => sha256:ac8ca584bf479ac0526ad5a97ecf822a669ac426b05538b4dfeb99696450acd2 30.13MB / 30.13MB                                                                                37.9s
 => => sha256:18749878119537de4d3cf5d4a8b634d56ed25af9a7bf1d9cf4c83d992bfae337 1.85MB / 1.85MB                                                                                  38.0s
 => => sha256:440e0189fc7a47c518761cd03079555af9fa07c112283436c304813f532a2f0b 22.07MB / 22.07MB                                                                                38.3s
 => => sha256:f85026953fe73d54370a7f96072b63d53603f86f4e150b7f015be6f262420b06 67.17MB / 67.17MB                                                                                38.7s
 => => sha256:160736157d2e712723b2eebc6efc35e185dedb7b6567422d0336ef694a66f7ad 161.23MB / 161.23MB                                                                              40.6s
 => => sha256:ca0ddd1a8c38af7b37d1130c66f0a50f332cfad0bb9d43a80f38a178af6aaeda 15.61MB / 15.61MB                                                                                40.8s
 => => sha256:162f79b5b388276b38f87a2fde699ab8d9106cf30d98584fc37909e9367ce10e 411.84MB / 411.84MB                                                                              43.4s
 => => sha256:bf5b40570db1be3247115c2cf3698fa6ae08a20f9856416ccb8b710d92e94d47 1.78GB / 1.78GB                                                                                  53.3s
 => => sha256:a25409264d46a2ce98edb8ec59308b7bc56fd8af86825741f6d7e2f401145e32 20.53kB / 20.53kB                                                                                53.4s
 => => sha256:e44523bf20109372c6636c82b13233e767d5e725e086704f4d08112690d4c6a6 495B / 495B                                                                                      53.4s
 => => sha256:b9ad851e1bef0c80fd03c06ebd62daba11ac4d79cb5d2d26acea2c0320474852 499B / 499B                                                                                      53.5s
 => => sha256:27872fc3573e1c2d159cd41a4db30d77bfe8ac1963b87567dc78ad58674d71a5 2.81kB / 2.81kB                                                                                  53.5s
 => ERROR [install_dependencies 1/7] FROM nvcr.io/nvidia/tritonserver:25.06-py3-min@sha256:c6b618aa65a360525e434012cd1cbd0ab576ca24f93ba14ea2165ea326739453                    281.9s
 => => resolve nvcr.io/nvidia/tritonserver:25.06-py3-min@sha256:c6b618aa65a360525e434012cd1cbd0ab576ca24f93ba14ea2165ea326739453                                                 0.0s
 => => sha256:8f997a27643b006907a2750d6b7754ba1bfe1303706df05d7252c74b6d4e27f5 4.52kB / 4.52kB                                                                                   0.0s
 => => sha256:6b644644f03e7903e75f249308d09df6b624d49ac97327058efe2c4b443ce0d4 24.16kB / 24.16kB                                                                                 0.0s
 => => sha256:f90c8eb4724ca5d2a796a06b736e16a72dae1ebb467ed19663c84d7d762d0a91 30.59MB / 30.59MB                                                                                 0.2s
 => => sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B / 32B                                                                                         0.1s
 => => sha256:cebaad6571a97399701575be85f05a33d3e7b5a0f24fc5473510a51e2aa673eb 3.83MB / 3.83MB                                                                                   0.1s
 => => sha256:c6b618aa65a360525e434012cd1cbd0ab576ca24f93ba14ea2165ea326739453 743B / 743B                                                                                       0.0s
 => => sha256:7848405aa76a398a6014ea511d9b0b52fb00dafa8e3e24ce3e0d85982fdaff16 89.88MB / 89.88MB                                                                                 0.5s
 => => sha256:1f7fdc9ac51458ab136ae88743d5bbfc0d2b57717f872349ff9ada895b600aea 16.91kB / 16.91kB                                                                                 0.3s
 => => sha256:3c313eb917ea07f6a2e8e8278cf840357ac5b88b3464f471efa90cedadc1e65a 114.38MB / 114.38MB                                                                               0.7s
 => => extracting sha256:f90c8eb4724ca5d2a796a06b736e16a72dae1ebb467ed19663c84d7d762d0a91                                                                                        0.8s
 => => sha256:d1c1b0ca7e576b38870258a7c0667ad16b820970c171ee36dc50ba041dc0f1dd 11.53kB / 11.53kB                                                                                 0.3s
 => => sha256:1d7d94b9f382ff484bf5c356f41fae712f2fa9a777d5f0ffe52f6e90833372e0 1.73MB / 1.73MB                                                                                   0.5s
 => => sha256:9f11a52127b5ce6c65cf99d5afa9fb7f5b0fe56d3f34c2b3ab4670e08259564b 179B / 179B                                                                                       0.4s
 => => sha256:f1af243d1e830ae7d4d0c27b2a39f13ebbd20eb3a64fc9ded54aada16af15d56 5.01kB / 5.01kB                                                                                   0.5s
 => => sha256:e21e46372e43c0c8ddd5593327607f990ed6c5ef7d9a81f4423e7b7a001edd69 216.26MB / 216.26MB                                                                               1.7s
 => => sha256:3f893d6c81004ca97381c46df634bc2bbc054b5b63e9831a8fa85984b51ca89d 6.37kB / 6.37kB                                                                                   0.5s
 => => sha256:47d8c16719752ae1120048af33336b3f82a7b613898c85e6f3981ae8d640cad2 106.39MB / 106.39MB                                                                               1.1s
 => => sha256:fa8ec00ab1d43016e10ba67c230b6a2c742b4c93ba344147ea75250ffe465db9 407B / 407B                                                                                       0.6s
 => => sha256:d260da74816ae1ec63adc9f90044c2a2e96f143450997fecc0a98eafc3bc0de5 4.71GB / 5.13GB                                                                                 282.0s
 => => extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1                                                                                        0.0s
 => => extracting sha256:cebaad6571a97399701575be85f05a33d3e7b5a0f24fc5473510a51e2aa673eb                                                                                        0.1s
 => => sha256:e327fded2fe59c3698beb0943cf7845ab5d7d5dba445e1d04b13aaba7424f0c6 35.96MB / 35.96MB                                                                                 1.3s
 => => extracting sha256:7848405aa76a398a6014ea511d9b0b52fb00dafa8e3e24ce3e0d85982fdaff16                                                                                        1.4s
 => => sha256:98d1cd1e264fb1e189debdfbcb78ae9aab91407e17b0a4d3515e347e0b8669df 503B / 503B                                                                                       1.3s
 => => sha256:57e194be082f9502172275266c867e10163bda932800b03f422d54504dd9833e 354.89MB / 354.89MB                                                                               3.2s
 => => sha256:0f33cb548b1619514df092c4f0ffffe4ae90dacedd9aa6ad7b18cdeaa8f6d7b1 232B / 232B                                                                                       1.7s
 => => sha256:16a101222e7a5ea7971d237fd399a15c195dd5b235ca00d85f9a509d70f47f38 750B / 750B                                                                                       1.7s
 => => extracting sha256:3c313eb917ea07f6a2e8e8278cf840357ac5b88b3464f471efa90cedadc1e65a                                                                                        1.5s
 => => extracting sha256:1f7fdc9ac51458ab136ae88743d5bbfc0d2b57717f872349ff9ada895b600aea                                                                                        0.0s
 => => extracting sha256:d1c1b0ca7e576b38870258a7c0667ad16b820970c171ee36dc50ba041dc0f1dd                                                                                        0.0s
 => => extracting sha256:9f11a52127b5ce6c65cf99d5afa9fb7f5b0fe56d3f34c2b3ab4670e08259564b                                                                                        0.0s
 => => extracting sha256:1d7d94b9f382ff484bf5c356f41fae712f2fa9a777d5f0ffe52f6e90833372e0                                                                                        0.0s
 => => extracting sha256:f1af243d1e830ae7d4d0c27b2a39f13ebbd20eb3a64fc9ded54aada16af15d56                                                                                        0.0s
 => => extracting sha256:3f893d6c81004ca97381c46df634bc2bbc054b5b63e9831a8fa85984b51ca89d                                                                                        0.0s
 => => extracting sha256:fa8ec00ab1d43016e10ba67c230b6a2c742b4c93ba344147ea75250ffe465db9                                                                                        0.0s
 => => extracting sha256:47d8c16719752ae1120048af33336b3f82a7b613898c85e6f3981ae8d640cad2                                                                                        2.3s
 => => extracting sha256:e21e46372e43c0c8ddd5593327607f990ed6c5ef7d9a81f4423e7b7a001edd69                                                                                        2.5s
------
 > [install_dependencies 1/7] FROM nvcr.io/nvidia/tritonserver:25.06-py3-min@sha256:c6b618aa65a360525e434012cd1cbd0ab576ca24f93ba14ea2165ea326739453:
------
Dockerfile.triton.trt_llm_backend:25
--------------------
  23 |     
  24 |     FROM ${PYTORCH_IMAGE} AS pytorch_image
  25 | >>> FROM ${BASE_IMAGE} AS install_dependencies
  26 |     
  27 |     WORKDIR /workspace
--------------------
Dockerfile.triton.trt_llm_backend:100
--------------------
  98 |     
  99 |     # Final stage to build the TRT-LLM container
 100 | >>> FROM ${BASE_IMAGE} AS final_stage
 101 |     
 102 |     ARG TORCH_VER
--------------------
ERROR: failed to solve: failed commit on ref "layer-sha256:d260da74816ae1ec63adc9f90044c2a2e96f143450997fecc0a98eafc3bc0de5": rename /var/lib/docker/buildkit/content/ingest/d9f7189c5496968ec24269229782518f4dba873b9ffdf10d4f7edbfa6443c9e3/data /var/lib/docker/buildkit/content/blobs/sha256/d260da74816ae1ec63adc9f90044c2a2e96f143450997fecc0a98eafc3bc0de5: no such file or directory

[achartier]

Could you try using https://github.com/NVIDIA/TensorRT-LLM/tree/main/docker with tritonrelease as the target stage? Note that it uses Triton 25.08, later releases of Triton have not been QA'd yet.

[geraldstanje1]

hi,

@achartier "Could you try using https://github.com/NVIDIA/TensorRT-LLM/tree/main/docker with tritonrelease as the target stage? Note that it uses Triton 25.08, later releases of Triton have not been QA'd yet." - that will not build triton inference server? Triton 25.08 have llm api 1.0?

my ultimate goal is to manually build triton inference server image with tensorrt llm 25.09 (which has llm api 1.0) - here i what i tried so far as well (to build triton inference server with tensorrt 25.09 - which has llm api 1.0)?

git clone https://github.com/triton-inference-server/server -b r25.09
cd server && ./build.py --backend tensorrtllm:release/1.0 --backend python:r25.09 --enable-gpu --build-type Release --target-platform
linux --endpoint grpc --endpoint http

getting error:
-- Found CUDA: /usr/local/cuda (found version "13.0")
-- Found Python3: /usr/bin/python3.12 (found version "3.12.3") found components: Interpreter Development Development.Module Development.Embed
CMake Error at CMakeLists.txt:151 (find_library):
Could not find tensorrt_llm using the following names: libtensorrt_llm.so

-- Configuring incomplete, errors occurred!
error: build failed

[achartier]

Once you have a TRT-LLM docker image, you could build Triton inside and replace the locally installed version. See these lines in the TRT-LLM docker build: https://github.com/NVIDIA/TensorRT-LLM/blob/main/docker/Dockerfile.multi#L99-L103

I'm not familiar with the Triton build system, you should ask directly on their github.

[geraldstanje1]

@achartier can we loop someone in who know about triton + tensorrtllm? it seems we run in a circle now - they say they dont know tensorrt llm...
what is this docker used for? https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/build.sh#L18

[achartier]

This docker is not used anymore, thanks for pointing it out. It has been removed in #813