Allow a content-based hash to be retrieved

[wouterbeek]

The hash would be based on the dictionary, the triples, and possibly the header. The content-based hash can for instance be used in order to verify that an HDT file contains the correct data.

[mielvds]

Also makes sense from an archiving perspective

[joernhees]

hmm, i also like the idea, but if you put it into the header then the hash will change, so where to put it? nested header??? ;)

[wouterbeek]

@joernhees The header also contains a timestamp, so it is relatively difficult to determine whether two HDT files contain the same data.

My idea was to store the triples-based hash and the dictionary-based hash in the header. Since the header is relatively small, its hash may be computed on the fly (without being stored).