Optimize Message Size: Datatypes

The necessary value range of the fields should be checked for potential optimization. The acceptance of the change will depend on the benchmark result.
A generic message should be created which contains most of the entries of GroundTruth and should be relistically filled. Afterwards the changes must me measured against the % in memory optimization.

[mbencik]

The indexing problem.

Each index at this point is an UINT64. The question is: Is there existsing a field that will have a need to index more than 4 billion elemenets in one repeated field? Is this expense justified?

The idea in the database indexing is to index the elements independent one type of another. For example. One SensorData (SD) can have multiple SensorViews (SV), that means that the SD gets a index 1. The SV is an repeated field but the indices start with 1 and continue. That gives us a situation that every SV is unique.

The problem arises if somebody decides to delete one SV. Lest say we have the SV 1, 2, 3.
There are now multiple solutions to that problem.
If some deletes the SV 2, all can stay as is but no more SV can be added to the message, so the next SV would be index 4. That has the advantage that the deleted elements can be tracked.
Option 2 the indices change with the deletion, so index 3 becomes index 2. That has the advantage that the indices are the same to the repeated field position, but we cannot track any data deletions.
Option 3 is to forbid deletion but to build in a flag that signal deletion or disabled status and use this. This has the advantage that all states of the data can be tracked and analyzed. But the data grow bigger if many states and changes are possible.

One further problem is that the index is not a field but a message. That creates a unique problem that for every index we have to dereference one pointer, and for all message member this is the case. That should be avoided.

[pmai]

Given that anything of this sort can only change in 4.0, and it is unclear to me whether 4.0 will even use ProtoBufs (see #56), I think micro-optimizations of this kind seem a bit premature and/or should be handled during #56 considerations.

[sebsonf]

For message size considerations please keep the protobuf wire format encoding in mind: https://developers.google.com/protocol-buffers/docs/encoding

Here, integers are represented using Varint encoding, so unsignificant bytes might get removed.