[TSan] Ignore reads if not stored early - #74575
Conversation
|
Should this be a runtime option, so that is can actively be en/disabled? |
As documented in this paper https://publications.rwth-aachen.de/record/840022/files/840022.pdf, we could trace back a significant runtime overhead introduced by certain HPC/scientific applications to concurrent shared read accesses. A typical scenario for such read accesses is matrix-vector multiplication which is frequently used to solve linar equation system. Accidentally, similar operations are also present in different machine learning algorithms. The performance issue typically arises when the code executes with more than 4 threads and gets worse when the threads are spread across different NUMA domains / sockets. The proposed change is to skip logging of reads, of they are not logged early. This means that previous reads by the current threads will still be updated. Empty shadow cells will also be used for logging. This change also avoids that previous writes get randomly overwritten by a read access. Under review as llvm#74575
|
If the 8-byte granule contains only 1 variable, or multiple uniformly accessed variables, then this may be OK as is, not sure completely. |
This looks like a property of region of memory, rather than property of a program. Think of e.g. ML model serving server that does both matmult and lots of generic server logic. I am also concerned about performance overhead of a global flag check in handling of every memory access. I am thinking is we could check that all existing accesses access the same byte range as the current read, then skipping the read looks more reasonable. This is still subject to some potential false negatives, which are extremely hard to discover later in real life (much worse than fixable false positives). So I thinking if we could also skip using some pseudo-random check. All of this needs benchmarking on real programs to estimate effects. |
3 participants
As documented in this paper https://publications.rwth-aachen.de/record/840022/files/840022.pdf, we could trace back a significant runtime overhead introduced by certain HPC/scientific applications to concurrent shared read accesses.
A typical scenario for such read accesses is matric-vector multiplication which is frequently used to solve linar equation system. Accidentally, similar operations are also present in different machine learning algorithms.
The performance issue typically arises when the code executes with more than 4 threads and gets worse when the threads are spread across different NUMA domains / sockets.
The proposed change is to skip logging of reads, of they are not logged early. This means that previous reads by the current threads will still be updated. Empty shadow cells will also be used for logging.
This change also avoids that previous writes get randomly overwritten by a read access.