[Enhancement]: Remove or optimize logtail TxnTable

[XuPeng-SH]

Is there an existing issue for the same feature request?

• I have checked the existing issues.

Is your feature request related to a problem?

The logtail TxnTable maintains a history of all transactions to support logtail collection via GetReader(from, to) and ScanInRange(). However, this design causes severe memory issues:

1. Memory accumulation: TxnBlocks retain all transaction history (memo/Tree) until checkpoint-based GC
2. OOM risk: ~0.93GB/min growth rate, TN crashes in ~19 minutes without checkpoint
3. Checkpoint dependency: GC only triggers after checkpoint completes, creating tight coupling

Root Cause

// pkg/vm/engine/tae/db/cronjobs.go
ckp := db.BGCheckpointRunner.MaxIncrementalCheckpoint()
if ckp == nil {
    return  // No checkpoint = No GC!
}

Memory breakdown from heap profile:

• TxnCtx/memo: 673MB (27%)
• Tree/TableRecord: 345MB (14%)
• mpool: 343MB (14%)

Describe the feature you'd like

Historical Context: This TxnTable design was needed when logtail collection required scanning transaction history. However, if the current logtail collection mechanism no longer needs this, we should remove or significantly optimize it.

Proposed Solution:

1. Verify if current logtail collection still needs TxnTable history
2. If not needed: Remove TxnTable entirely or replace with lightweight tracking
3. If still needed: Decouple GC from checkpoint with time-based or memory-pressure cleanup

Implementation Considered

Key files to modify:

• pkg/vm/engine/tae/logtail/table.go - TxnTable/txnBlock implementation
• pkg/vm/engine/tae/logtail/mgr.go - GC logic
• pkg/vm/engine/tae/db/cronjobs.go - GC trigger

Use Cases

• High-load TN deployments without frequent checkpoints
• Long-running transactions scenarios
• Memory-constrained environments

Additional information

Impact:

• Eliminates major memory leak source in TN
• Removes checkpoint dependency for logtail memory management
• Improves system stability under high load