Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix copy of tensors for TRT on iGPU (triton-inference-server#3067)
* Add logging * Leverage zero copy support for output tensor - When possible * Wait for inference completion on iGPU to correctly copy output tensors * Add missing quote in error msg * Remove debug logging * Fix synchronization bug * Remove extra logging * Use cudaLaunchHostFunc * Re-use zero copy, make class variable * Fix sync * review edits * Avoid invalidation of vector element address * fix sync and lifecycle of payload * Use struct to store params * cleanup * Update comments * use resize instead of reserve * direct cuda host launch in copybuffer * Update copyright * Remove extra cudaStreamWaitEvent * Use memcpy on stream for inputs * review edits * add temp logging * io_binding_info was already set earlier * additional review edits * Remove redundant block of code * Fix logging after SupportsIntegratedZeroCopy()
- Loading branch information