fix(db): Fix a sprout/history tree read panic in Zebra 1.4.0, which only happens before the 25.3.0 state upgrade completes

[teor2345]

Motivation

Zebra 1.4.0 only writes the sprout and history trees when they have changed, but it expects to find them at the empty key () or tip height. If there aren't any sprout transactions, or we're before heartwood, the trees can actually be below the tip height.

See this detailed explanation:
#7961 (comment)

There is also a race condition that can happen when upgrading from 1.1.0 to 1.4.0, which we support on a best effort basis.

See this detailed explanation for this possible race condition:
#7961 (comment)

Close #7961.

PR Author Checklist

Check before marking the PR as ready for review:

• Will the PR name make sense to users?
• Does the PR have a priority label?
• Have you added or updated tests?
• Is the documentation up to date?

For significant changes:

• Is there a summary in the CHANGELOG?
• Can these changes be split into multiple PRs?

If a checkbox isn't relevant to the PR, mark it as done.

We might want a changelog entry, I'm not sure.

Complex Code or Requirements

This is concurrent code involving a data race.

Solution

• Look for the sprout tree at the greatest available height before and during the 25.3.0 upgrade, rather than just checking the tip height
• Minimise the risk of a potential data race before and during the 25.3.0 state upgrade, by moving slow code to another database batch

Testing

Marek and the ticket reporter have both manually tested this code and confirmed it is correct.

I don't think we can test for the race condition that's fixed in this PR.

Review

This is a high priority fix.

Reviewer Checklist

Check before approving the PR:

• Does the PR scope match the ticket?
• Are there enough tests to make sure it works? Do the tests cover the PR motivation?
• Are all the PR blockers dealt with?
  PR blockers can be dealt with in new tickets or PRs.

And check the PR Author checklist is complete.

Follow Up Work

Document how to avoid this happening again in the state upgrade docs.

In particular, don't concurrently read and write the same keys in the same column family:

• in an upgrade, and
• in the sync.

And make sure the read and write methods have compatible conditions.