Make rmm::prefetch fault tolerant #1649
Conversation
github-actions
bot
added
Python
bdice
added
feature request
| @@ -33,6 +33,11 @@ namespace rmm::mr { | |||
| /** | |||
| * @brief Resource that prefetches all memory allocations. | |||
| * | |||
| * Note that prefetching is a no-op if the upstream resource is not using | |||
There was a problem hiding this comment.
This adaptor has more information than rmm::prefetch does. It could (in its ctor) detect that the upstream MR is not managed memory, or that the device doesn't support managed memory. Then it could skip the prefetching, saving a call to cudaMemPrefetchAsync on every alloc.
If it doesn't do this, then there is overhead on alloc on systems that don't support prefetching, and so this isn't truly a no-op and the docs are technically inaccurate.
I think it would be better to make it truly a no-op. What do you think @bdice ?
There was a problem hiding this comment.
I agree 100% and want to do that! I did not implement this because I did not know how to check if the upstream MR is using managed memory. Do you have advice on how to do that?
There was a problem hiding this comment.
Do a test allocation, try to prefetch it, check for errors, deallocate it.
Description
Closes #1648.
Checklist