Skip to content

[TRT EP] Fix trt context memory sharing #24784

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 20, 2025

Conversation

toothache
Copy link
Member

@toothache toothache commented May 16, 2025

Description

Fixed a TRT context memory sharing bug where the context memory was assigned to a unique_ptr that was immediately destructed upon leaving scope.

Motivation and Context

The bug seems to be introduced by a refactor work: #15833 :

image

@toothache
Copy link
Member Author

@yf711 @chilo-ms , may I get a review for this change? Thanks!

@jywu-msft jywu-msft requested a review from chilo-ms May 20, 2025 02:50
@tianleiwu
Copy link
Contributor

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@chilo-ms
Copy link
Contributor

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@jywu-msft jywu-msft merged commit adeb016 into microsoft:main May 20, 2025
82 checks passed
@jywu-msft
Copy link
Member

@chilo-ms can you help double check if this same issue is there on NV RTX EP?

@chilo-ms
Copy link
Contributor

@chilo-ms can you help double check if this same issue is there on NV RTX EP?

@ankan-ban @ishwar-raut1 @gedoensmax @gaugarg-nv
Please see this TRT context memory sharing bug and fix in TRT EP.
Even though NV RTX EP for now doesn't expose TRT context memory sharing option, you might need the fix later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants