Explore using inline data storage

[rklaehn]

The most common hashes are 32 bytes long. So with the additional data that gives 34 bytes. It seems inefficient to store such a small object on the heap with the associated overhead / pointer chasing / cache issues.

It would be possible to use an approach similar to https://crates.io/crates/smallvec and store hashes inline up to a certain size, and only store on the heap once the needed size is bigger.

Note that we would not want to use smallvec since we don't need all the metadata of a vec. The hash already knows how big it is, so what we would need would be just an enum like this:

enum Storage {
  Inline([u8; 39]), // any hash up to 39 bytes goes in here
  Heap(Arc<[u8]>), // the rare hash that is larger goes in here
}

This can be made pretty much opaque from the outside, like smallvec. Might not even need unsafe...

[rklaehn]

Overall size would be 40 bytes, still smaller than the typical cache line size.

[vmx]

That improvement got merged.