Add internal/btree: in-memory B+Tree with search, insert and split

[suhailopensource]

Summary

Adds internal/btree: an in-memory B+Tree with search, insert and node
splitting. No disk involvement, so the algorithm is the only thing under test.

This is the first structure in the project that makes "find this key" fast.

Why a tree

Measured over 100,000 keys:

structure     find one key   range query   problem
linear scan   ~60,000 ns     yes           reads everything
hash map      5-10 ns        NO            no order at all
binary tree   O(log n)       yes           one key per node
B+Tree        55-60 ns       yes           -

Ranges across three runs, not single figures. A hash map is roughly 8x faster
than the B+Tree at finding a single key. It is also
completely useless for WHERE id BETWEEN 200 AND 300, because hashing destroys
order. That one column is why databases index with trees rather than hash
tables, and it is worth stating plainly: the B+Tree is not the fastest way to
look up one key, it is the fastest way to look up one key while keeping the
ability to walk in order.

A plain binary tree keeps order too, but holds one key per node. At 100,000 keys
that is ~17 levels; a B+Tree at order 64 is 4. On disk each level is a page
read, so the fanout is the entire point.

Design

Internal nodes hold separators, leaves hold values

This is the difference between a B-Tree and a B+Tree. A B-Tree stores values in
internal nodes too; a B+Tree keeps every value in a leaf and uses internal nodes
purely as signposts.

internal [070]
    internal [030 050]
        leaf     [010 020]
        leaf     [030 040]
        leaf     [050 060]
    internal [090 110]
        leaf     [070 080]
        leaf     [090 100]
        leaf     [110 120]

The payoff arrives with range scans: once leaves are chained, a range is a walk
along the bottom row rather than a repeated descent.

The two splits behave differently

Easy to get wrong, and the tests pin both:

• splitLeaf copies its middle key upward. The key still has a value
  attached, so it must remain in the leaf as well as appearing as a separator.
• splitInternal moves its middle key upward. It is only a separator, so
  keeping a copy would be duplication.

When the root splits, a new root is created above it. That is the only way the
tree gets taller, which is why all leaves stay at the same depth with no
rebalancing pass — growth happens at the root, not at the leaves.

Two different binary searches

childIndex and leafIndex are not the same function, which is a classic
source of off-by-one bugs:

• descending an internal node wants the first index where key < keys[i]
• searching a leaf wants the first index where keys[i] >= key

They are separate methods rather than one with a flag, so a caller cannot pick
the wrong one silently.

Keys are copied on insert

Put clones the key. A caller reusing a buffer across inserts would otherwise
see every stored key mutate together. This is the same defect class found in
codec.Bytes during the 0x04 review, so it is guarded by a test here:
TestKeysAreCopiedNotAliased writes a key, mutates the caller's buffer, and
checks the tree is unaffected.

Height maths

order   level 2      level 3        level 4
64      4,032        258,048        16,515,072
256     65,280       16,711,680     4,278,190,080
512     261,632      133,955,584    68,585,259,008

At order 512 a four-level tree addresses over 68 billion keys. Height grows like
log(n), so multiplying the data by 500 adds one level. Measured: 100,000 keys at
order 64 built in 18ms, height 4, lookups ~55 ns and inserts ~110 ns — insertion
costing roughly twice a lookup is the price of shifting keys within a node.

The zero value is usable

An unconfigured Tree reads as empty — Get misses, Height is 0, Validate
passes, Dump is empty — and initialises itself at DefaultOrder (64) on first
write. Before this it dereferenced a nil root and panicked.

Get pays one nil check for this; the benchmark is unchanged. This is the same
defect class as the zero-value Layout in the previous step, so it was probed
for deliberately rather than found by accident.

Testing

Validate() is a first-class method rather than a test helper, because a
storage engine benefits from being able to check its own structure. It asserts:

• keys sorted within every node
• every key inside the bounds implied by its ancestors' separators
• all leaves at the same depth
• len(children) == len(keys) + 1 in internal nodes
• no node over order-1 keys; no non-root node under (order-1)/2
• the root is a leaf or has at least two children

The bounds check is the one that catches real bugs: a split that promotes the
wrong key produces a tree that still looks locally sorted but sends searches to
the wrong leaf.

TestInvariantsHoldAfterEveryInsert runs Validate() after each of 500
random inserts, across orders 3, 4, 5, 8 and 32. TestSortedAndReverseInsertion
covers the two worst cases for a naive split, since ascending inserts always
split the rightmost node.

Two fuzz targets, 20s each, clean:

• FuzzInsertKeepsInvariants — arbitrary order and key sequence; validates after
  every insert and checks every inserted key is still findable
• FuzzGetNeverPanics — arbitrary probe keys against a populated tree

Defect found by adversarial probing of this change, now covered by
TestZeroValueTreeIsUsable: a zero-value Tree panicked with a nil pointer
dereference on Get, Height, Validate and Dump.

Five further probes found nothing, and are kept as regression tests:

• TestLeavesFormOneSortedSequence — the strongest check here. Walks the leaves
  left to right across orders 3, 4, 7 and 16 after 2,000 random inserts each,
  and asserts the result is one complete, strictly sorted sequence. The
  per-node invariants can all hold while the tree as a whole is wrong; this
  cannot.
• TestPrefixKeys — keys where one is a prefix of another (a, aa, aaa,
  ab, ""), the case most likely to break a comparison
• TestNoPhantomKeys — inserting evens must not make odds findable
• TestOverwriteAcrossSplits — overwrite all 300 keys after the tree is deep
• TestNilAndEmptyKeysAreTheSameKey — nil and []byte{} must not become two
  different keys

72 tests across the module. go vet, gofmt -l, go test -race ./... clean.

The trap this does not solve

The tree compares keys with bytes.Compare and nothing else. It has no idea a
key is a number, so a wrong encoding produces wrong answers and never errors:

as text:                        [100 110 120 20 30]
via codec.AppendKeyUint64:      [20 30 100 110 120]

The demo shows this deliberately, because a correct tree over a bad key encoding
is exactly the kind of failure that survives a full test suite.

Not in scope

• Leaf sibling pointers and range scans. The chained-leaf walk is the reason
  to prefer a B+Tree, but it is a separate change and touches splitLeaf.
• Deletion. Borrowing and merging are harder than splitting, and the
  minimum-keys invariant is already asserted in anticipation.
• index.Index. btree provides Get and Put but does not claim the
  interface until Scan and Delete exist. Declaring it early would be a lie
  the compiler cannot catch.
• Pages and disk. Nodes are Go structs with pointers. Turning them into page
  reads is a later step, and doing it now would mean debugging two things at
  once.
• Duplicate keys. Put overwrites. Non-unique indexes are their own step.
• Concurrency. No locking; a tree under concurrent writers is undefined.