You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Single-threaded datacheck FP matching is already pretty efficient:
Currently roughly 2/3 of datacheck entries are FPs.
datacheckerrors->entries is sorted asciibetically, and datacheckfps.csv is sorted pretty darn close to asciibetically.
Thus when an FP is matched, it's found & erased right off the beginning of the FP list, very little searching necessary.
When there's no FP entry to match, we do search the whole list, but at least it's continually shrinking.
Still, it's about 5% of total execution time on lab2.
It doesn't lend itself well to multithreading:
How to avoid data races when reading & erasing from the datacheckfps list?
We could use a mutex, but the performance penalty would probably be a big one.
Attempting to avoid this by splitting datacheckerrors->entries and datacheckfps into n chunks to feed to n threads hurts our ability to "cross off" items right away.
A better approach?
Some datacheck errors are flagged during HighwayGraph construction, but after the structure is built out, we're all set.
Rather than lose efficiency with any of the above approaches, we can process datacheck FPs in the background as we begin to write subgraphs.
Spawn one less SubgraphThread to start, and have our datacheck FP thread spawn one when it finishes up, just like MasterTmgThread does.
This will require getting a bit more clever when threading is enabled but -t 1 is specified.
The text was updated successfully, but these errors were encountered:
Single-threaded datacheck FP matching is already pretty efficient:
datacheckerrors->entries
is sorted asciibetically, and datacheckfps.csv is sorted pretty darn close to asciibetically.Still, it's about 5% of total execution time on lab2.
It doesn't lend itself well to multithreading:
datacheckfps
list?datacheckerrors->entries
anddatacheckfps
into n chunks to feed to n threads hurts our ability to "cross off" items right away.A better approach?
Some datacheck errors are flagged during HighwayGraph construction, but after the structure is built out, we're all set.
SubgraphThread
to start, and have our datacheck FP thread spawn one when it finishes up, just likeMasterTmgThread
does.-t 1
is specified.The text was updated successfully, but these errors were encountered: