Skip to content

Commit

Permalink
Clarify that chunker sizes are in bytes
Browse files Browse the repository at this point in the history
Clarify that the Rabin fingerprint chunker is in bytes, and recommend
using larger chunk sizes than shown in the provided examples.

People are getting confused over chunker sizes and incorrectly assuming
they're in kiB and not bytes.
  • Loading branch information
da2x committed Jan 13, 2019
1 parent b36037a commit 9dc125e
Showing 1 changed file with 9 additions and 5 deletions.
14 changes: 9 additions & 5 deletions core/commands/add.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,16 @@ You can now refer to the added file in a gateway, like so:
The chunker option, '-s', specifies the chunking strategy that dictates
how to break files into blocks. Blocks with same content can
be deduplicated. The default is a fixed block size of
be deduplicated. Different chunking strategies will produce different
hashes for the same file. The default is a fixed block size of
256 * 1024 bytes, 'size-262144'. Alternatively, you can use the
rabin chunker for content defined chunking by specifying
rabin-[min]-[avg]-[max] (where min/avg/max refer to the resulting
chunk sizes). Using other chunking strategies will produce
different hashes for the same file.
Rabin fingerprint chunker for content defined chunking by specifying
rabin-[min]-[avg]-[max] (where min/avg/max refer to the desired
chunk sizes in bytes), e.g. 'rabin-262144-524288-1048576'.
The following examples use very small byte sizes to demonstrate the
properties of the different chunkers on a small file. You'll likely
want to use a 1024 times larger chunk sizes for most files.
> ipfs add --chunker=size-2048 ipfs-logo.svg
added QmafrLBfzRLV4XSH1XcaMMeaXEUhDJjmtDfsYU95TrWG87 ipfs-logo.svg
Expand Down

1 comment on commit 9dc125e

@GitCop
Copy link

@GitCop GitCop commented on 9dc125e Jan 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were the following issues with your Pull Request

We ask for a few features in the commit message for Open Source licensing hygiene and commit message clarity.
git commit --amend can often help you quickly improve the commit message.
Guidelines and a script are available to help in the long run.
Your feedback on GitCop is welcome on this issue.


This message was auto-generated by https://gitcop.com

Please sign in to comment.