Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give TensorRT-LLMa proper CI/CD 😍 #2886

Merged
merged 134 commits into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
134 commits
Select commit Hold shift + click to select a range
f729f2c
test(ctest) enable address sanitizer
mfuntowicz Nov 18, 2024
0baa017
feat(trtllm): expose finish reason to Rust
mfuntowicz Dec 10, 2024
cb8fdde
feat(trtllm): fix logits retrieval
mfuntowicz Dec 10, 2024
0ab1dd8
misc(ci): enabe building tensorrt-llm
mfuntowicz Dec 12, 2024
119a40c
misc(ci): update Rust action toolchain
mfuntowicz Dec 12, 2024
7db90f1
misc(ci): let's try to build the Dockerfile for trtllm
mfuntowicz Dec 12, 2024
3f8dc96
misc(ci): provide mecanism to cache inside container
mfuntowicz Dec 12, 2024
0aa49a1
misc(ci): export aws creds as output of step
mfuntowicz Dec 12, 2024
ea7cf3a
misc(ci): let's try this way
mfuntowicz Dec 12, 2024
bdab3bb
misc(ci): again
mfuntowicz Dec 12, 2024
dc34f5a
misc(ci): again
mfuntowicz Dec 12, 2024
f939500
misc(ci): add debug profile
mfuntowicz Dec 12, 2024
2737416
misc(ci): add debug profile
mfuntowicz Dec 12, 2024
b43fe7e
misc(ci): lets actually use sccache ...
mfuntowicz Dec 12, 2024
55c92d0
misc(ci): do not build with ssl enabled
mfuntowicz Dec 12, 2024
88884f9
misc(ci): WAT
mfuntowicz Dec 12, 2024
5fbab27
misc(ci): WAT
mfuntowicz Dec 12, 2024
425f0bf
misc(ci): WAT
mfuntowicz Dec 12, 2024
ba738e2
misc(ci): WAT
mfuntowicz Dec 13, 2024
253116e
misc(ci): WAT
mfuntowicz Dec 13, 2024
5d5524d
misc(backend): test with TGI S3 conf
mfuntowicz Dec 16, 2024
f1986c0
misc(backend): test with TGI S3 conf
mfuntowicz Dec 16, 2024
783a057
misc(backend): once more?
mfuntowicz Dec 16, 2024
71311be
misc(backend): let's try with GHA
mfuntowicz Dec 17, 2024
fd039b6
misc(backend): missing env directive
mfuntowicz Dec 17, 2024
7f9b223
misc(backend): make sure to correctly set IS_GHA_BUILD=true in wf
mfuntowicz Dec 17, 2024
b8d755e
misc(backend): ok let's debug smtg
mfuntowicz Dec 17, 2024
d0108b4
misc(backend): WWWWWWWWWWWWWAAAAAAAA
mfuntowicz Dec 17, 2024
6d4ac29
misc(backend): kthxbye retry s3
mfuntowicz Dec 17, 2024
7337d83
misc(backend): use session token
mfuntowicz Dec 17, 2024
4394a23
misc(backend): add more info
mfuntowicz Dec 17, 2024
b5c62c4
misc(backend): lets try 1h30
mfuntowicz Dec 17, 2024
76239f2
misc(backend): lets try 1h30
mfuntowicz Dec 17, 2024
84ea221
misc(backend): increase to 2h
mfuntowicz Dec 18, 2024
656dc23
misc(backend): lets try...
mfuntowicz Dec 18, 2024
da4bd56
misc(backend): lets try...
mfuntowicz Dec 18, 2024
7a1785f
misc(backend): let's build for ci-runtime
mfuntowicz Dec 19, 2024
aa6a143
misc(backend): let's add some more tooling
mfuntowicz Dec 19, 2024
06fb820
misc(backend): add some tags
mfuntowicz Dec 19, 2024
4aae931
misc(backend): disable Werror for now
mfuntowicz Dec 19, 2024
724e0c1
misc(backend): added automatic gha detection
mfuntowicz Dec 19, 2024
7f6b1f1
misc(backend): remove leak sanitizer which is included in asan
mfuntowicz Dec 20, 2024
d1a9318
misc(backend): forward env
mfuntowicz Dec 20, 2024
ebb3e51
misc(backend): forward env
mfuntowicz Dec 20, 2024
8609f0d
misc(backend): let's try
mfuntowicz Dec 20, 2024
ffbab2c
misc(backend): let's try
mfuntowicz Dec 20, 2024
f226b53
misc(backend): again
mfuntowicz Dec 20, 2024
8bcfe5a
misc(backend): again
mfuntowicz Dec 20, 2024
90bc544
misc(backend): again
mfuntowicz Dec 20, 2024
0383617
misc(backend): again
mfuntowicz Dec 20, 2024
30ed776
misc(backend): again
mfuntowicz Dec 20, 2024
cf7069d
misc(backend): fix sscache -> sccache
mfuntowicz Dec 20, 2024
bb77ae9
misc(backend): fix sscache -> sccache
mfuntowicz Dec 20, 2024
3cd97a3
misc(backend): fix sscache -> sccache
mfuntowicz Dec 20, 2024
2556626
misc(backend): let's actually cache things now
mfuntowicz Dec 20, 2024
62d5ade
misc(backend): let's actually cache things now
mfuntowicz Dec 20, 2024
aa3b2d5
misc(backend): attempt to run the testS?
mfuntowicz Dec 21, 2024
e128922
misc(backend): attempt to run the tests?
mfuntowicz Dec 21, 2024
a0a9534
misc(backend): attempt to run the tests?
mfuntowicz Dec 21, 2024
2d7cd0e
change runner size
glegendre01 Dec 27, 2024
356eff9
fix: Correctly tag docker images (#2878)
Hugoch Jan 6, 2025
7f10191
misc(llamacpp): maybe?
mfuntowicz Jan 6, 2025
d497533
misc(llamacpp): maybe?
mfuntowicz Jan 6, 2025
0c25993
misc(llamacpp): maybe?
mfuntowicz Jan 6, 2025
0dcbe96
misc(ci): gogogo
mfuntowicz Jan 6, 2025
5429a11
misc(ci): gogogo
mfuntowicz Jan 6, 2025
6d1bd37
misc(ci): gogogo
mfuntowicz Jan 6, 2025
77ffe9d
misc(ci): gogogo
mfuntowicz Jan 6, 2025
4c2c3aa
misc(ci): gogogo
mfuntowicz Jan 6, 2025
77cbd65
misc(ci): gogogo
mfuntowicz Jan 6, 2025
c218e3d
misc(ci): go
mfuntowicz Jan 6, 2025
918c3ad
misc(ci): go
mfuntowicz Jan 6, 2025
92cfe43
misc(ci): go
mfuntowicz Jan 6, 2025
dc0fd7b
misc(ci): use bin folder
mfuntowicz Jan 6, 2025
228b3d6
misc(ci): make the wf callable for reuse
mfuntowicz Jan 7, 2025
b7e848e
misc(ci): make the wf callable for reuse (bis)
mfuntowicz Jan 7, 2025
d5224b3
misc(ci): make the wf callable for reuse (bis)
mfuntowicz Jan 7, 2025
4266d9e
misc(ci): give the wf a name
mfuntowicz Jan 7, 2025
3ef5e02
Create test-trtllm.yml
paulinebm Jan 7, 2025
994f0ab
Update test-trtllm.yml
paulinebm Jan 7, 2025
5d3d8c0
Create build-trtllm2
paulinebm Jan 7, 2025
29ac684
Rename build-trtllm2 to 1-build-trtllm2
paulinebm Jan 7, 2025
38f226f
Rename test-trtllm.yml to 1-test-trtllm2.yml
paulinebm Jan 7, 2025
e37b7f7
misc(ci): fw secrets
mfuntowicz Jan 7, 2025
42f0296
Update 1-test-trtllm2.yml
paulinebm Jan 7, 2025
327cb48
Rename 1-build-trtllm2 to 1-build-trtllm2.yml
paulinebm Jan 7, 2025
0d83c00
Update 1-test-trtllm2.yml
paulinebm Jan 7, 2025
9b87d1a
misc(ci): use ci-build.yaml as main dispatcher
mfuntowicz Jan 7, 2025
1736086
Delete .github/workflows/1-test-trtllm2.yml
paulinebm Jan 7, 2025
d47ce65
Delete .github/workflows/1-build-trtllm2.yml
paulinebm Jan 7, 2025
5e8fdd3
misc(ci): rights?
mfuntowicz Jan 7, 2025
b3277a3
misc(ci): rights?
mfuntowicz Jan 7, 2025
77e42c2
misc(ci): once more?
mfuntowicz Jan 7, 2025
f6d5f71
misc(ci): once more?
mfuntowicz Jan 7, 2025
215fad0
misc(ci): baby more time?
mfuntowicz Jan 7, 2025
5a73fe5
misc(ci): baby more time?
mfuntowicz Jan 7, 2025
7da7b38
misc(ci): try the permission above again?
mfuntowicz Jan 8, 2025
e80ce22
misc(ci): try the permission above again?
mfuntowicz Jan 8, 2025
b3fae2f
misc(ci): try the permission scoped again?
mfuntowicz Jan 8, 2025
b35a14b
misc(ci): install tensorrt_llm_executor_static
mfuntowicz Jan 8, 2025
c5aa514
misc(ci): attempt to rebuild with sccache?
mfuntowicz Jan 8, 2025
df9df1d
misc(ci):run the tests on GPU instance
mfuntowicz Jan 8, 2025
a7a7c67
misc(ci): let's actually setup sccache in the build.rs
mfuntowicz Jan 8, 2025
4d875c4
misc(ci): reintroduce variables
mfuntowicz Jan 9, 2025
7bde6d3
misc(ci): enforce sccache
mfuntowicz Jan 9, 2025
d110ab2
misc(ci): correct right job name dependency
mfuntowicz Jan 9, 2025
75b9d82
misc(ci): detect dev profile for debug
mfuntowicz Jan 9, 2025
7a893af
misc(ci): detect gha build
mfuntowicz Jan 9, 2025
1e08e9c
misc(ci): detect gha build
mfuntowicz Jan 9, 2025
a791291
misc(ci): ok debug
mfuntowicz Jan 9, 2025
af6428c
misc(ci): wtf
mfuntowicz Jan 9, 2025
c3f3035
misc(ci): wtf2
mfuntowicz Jan 9, 2025
8f0da40
misc(ci): wtf3
mfuntowicz Jan 10, 2025
3c7710c
misc(ci): use commit HEAD instead of merge commit for image id
mfuntowicz Jan 10, 2025
79469be
misc(ci): wtfinfini
mfuntowicz Jan 10, 2025
0cf0732
misc(ci): wtfinfini
mfuntowicz Jan 10, 2025
0159843
misc(ci): KAMEHAMEHA
mfuntowicz Jan 10, 2025
d969dad
Merge TRTLLM in standard CI
Hugoch Jan 15, 2025
ffb60ff
misc(ci): remove input machine
mfuntowicz Jan 20, 2025
b267df5
misc(ci): missing id-token for AWS auth
mfuntowicz Jan 20, 2025
e083a92
misc(ci): missing id-token for AWS auth
mfuntowicz Jan 20, 2025
87039cf
misc(ci): missing id-token for AWS auth
mfuntowicz Jan 20, 2025
549d7e3
misc(ci): again...
mfuntowicz Jan 20, 2025
c33eeb2
misc(ci): again...
mfuntowicz Jan 20, 2025
0a33615
misc(ci): again...
mfuntowicz Jan 20, 2025
08af269
misc(ci): again...
mfuntowicz Jan 20, 2025
0ed76ba
misc(ci): missing benchmark
mfuntowicz Jan 20, 2025
8c6d972
misc(ci): missing backends
mfuntowicz Jan 20, 2025
debaea4
misc(ci): missing launcher
mfuntowicz Jan 20, 2025
7c9ee56
misc(ci): give everything aws needs
mfuntowicz Jan 20, 2025
d0b8e2e
misc(ci): give everything aws needs
mfuntowicz Jan 20, 2025
edfafeb
misc(ci): fix warnings
mfuntowicz Jan 20, 2025
a4d069f
misc(ci): attempt to fix sccache not building trtllm
mfuntowicz Jan 20, 2025
a0e75b1
misc(ci): attempt to fix sccache not building trtllm again
mfuntowicz Jan 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
misc(backend): once more?
  • Loading branch information
mfuntowicz committed Jan 20, 2025
commit 783a057ac01e3c7f436dfbadef827d196f9c321b
8 changes: 3 additions & 5 deletions .github/workflows/build_trtllm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,12 @@ jobs:
SCCACHE_REGION: us-east-1
SCCACHE_S3_KEY_PREFIX: ${{ env.TENSORT_LLM_VERSION }}
# SCCACHE_GHA_ENABLED=true

secrets:
# ACTIONS_CACHE_URL=${{}}
# ACTIONS_RUNTIME_TOKEN=${{}}

AWS_ACCESS_KEY_ID=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_SECRET_ACCESS_KEY }}
SCCACHE_BUCKET=ci-docker-buildx-cache,name=text-generation-inference-cache-${{ env.LABEL }}
aws_access_key_id=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_ACCESS_KEY_ID }}
aws_secret_access_key=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_SECRET_ACCESS_KEY }}
sccache_aws_bucket_name=ci-docker-buildx-cache,name=text-generation-inference-cache-${{ env.LABEL }}
cache-from: type=s3,region=us-east-1,bucket=ci-docker-buildx-cache,name=text-generation-inference-cache-${{ env.LABEL }},mode=min,access_key_id=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_ACCESS_KEY_ID }},secret_access_key=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_SECRET_ACCESS_KEY }},mode=min
cache-to: type=s3,region=us-east-1,bucket=ci-docker-buildx-cache,name=text-generation-inference-cache-${{ env.LABEL }},mode=min,access_key_id=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_ACCESS_KEY_ID }},secret_access_key=${{ secrets.S3_CI_DOCKER_BUILDX_CACHE_SECRET_ACCESS_KEY }},mode=min

Expand Down
6 changes: 3 additions & 3 deletions Dockerfile_trtllm
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,9 @@ COPY router router
COPY backends/trtllm backends/trtllm
COPY --from=trt-builder /usr/local/tensorrt /usr/local/tensorrt
COPY --from=mpi-builder /usr/local/mpi /usr/local/mpi
RUN --mount=type=secret,id=AWS_ACCESS_KEY_ID,env=AWS_ACCESS_KEY_ID \
--mount=type=secret,id=AWS_SECRET_ACCESS_KEY,env=AWS_SECRET_ACCESS_KEY \
--mount=type=secret,id=SCCACHE_BUCKET,env=SCCACHE_BUCKET \
RUN --mount=type=secret,id=aws_access_key_id,env=AWS_ACCESS_KEY_ID \
--mount=type=secret,id=aws_secret_access_key,env=AWS_SECRET_ACCESS_KEY \
--mount=type=secret,id=sccache_aws_bucket_name,env=SCCACHE_BUCKET \
mkdir $TGI_INSTALL_PREFIX && mkdir "$TGI_INSTALL_PREFIX/include" && mkdir "$TGI_INSTALL_PREFIX/lib" && \
RUSTC_WRAPPER=sccache CMAKE_INSTALL_PREFIX=$TGI_INSTALL_PREFIX cargo build --profile ${BUILD_TYPE} --package text-generation-backends-trtllm --bin text-generation-backends-trtllm

Expand Down