feat(file): parallelize file hashing #347
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #321
Applied changes based on Issue #321.
I’ve parallelized the file-hashing loop in
captureFileState
so that files are hashed in parallel—capped to the number of CPU cores—rather than one by one.Summary of changes:
os
module to determineos.cpus().length
for a sensible concurrency cap.for…of
loop with a pool of “worker” tasks that each pull the next file to hash from a shared index.fs.stat()
andcalculateFileHash()
inside a try/catch, preserving the existing warning behavior.Promise.all
, then return the fullfileState
map as before.This should significantly speed up
captureFileState
on larger workspaces without changing its external behavior or test results. Let me know if you’d like further tweaks!