Skip to content

copyFile with bufferSize instead of hardcoded value #22769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 75 commits into from
Sep 30, 2023
Merged

copyFile with bufferSize instead of hardcoded value #22769

merged 75 commits into from
Sep 30, 2023

Conversation

juancarlospaco
Copy link
Collaborator

@juancarlospaco juancarlospaco commented Sep 28, 2023

  • copyFile allows to specify bufferSize instead of hardcoded wrong value. Tiny diff.

Performance

  • 1200% Performance improvement.

Check it yourself

Execute:

for i in $(seq 0 10); do
  bs=$((1024*2**$i))
  printf "%7s Kb\t" $bs
  timeout --foreground -sINT 2 dd bs=$bs if=/dev/zero of=/dev/null 2>&1 | sed -n 's/.* \([0-9.,]* [GM]B\/s\)/\1/p'
done

Stats

  • Hardcoded 8192 or 8000 Kb bufferSize gives 5 GB/s.
  • Setting 262144 Kb bufferSize gives 65 GB/s (script suggestion).

@juancarlospaco juancarlospaco marked this pull request as ready for review September 29, 2023 01:12
@Araq
Copy link
Member

Araq commented Sep 29, 2023

No, not a new overload, just add a parameter with a default value.

@juancarlospaco juancarlospaco marked this pull request as draft September 29, 2023 10:32
@juancarlospaco juancarlospaco marked this pull request as ready for review September 30, 2023 00:45
@Araq Araq merged commit a38e3dc into nim-lang:devel Sep 30, 2023
@github-actions
Copy link
Contributor

Thanks for your hard work on this PR!
The lines below are statistics of the Nim compiler built from a38e3dc

Hint: mm: orc; opt: speed; options: -d:release
170093 lines; 10.322s; 620.043MiB peakmem

Araq added a commit that referenced this pull request Oct 1, 2023
- Continuation of #22769
- See
https://pubs.opengroup.org/onlinepubs/9699919799/functions/posix_fadvise.html
- The code was already there in `std/posix` since years ago. 3 line
diff.

---------

Co-authored-by: Andreas Rumpf <rumpf_a@web.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants