changefeedccl: restore checkpoint info to changeFrontier on resume

[andyyang890]

When we resume a changefeed, we only pass the checkpoint to the changefeed aggregators, not the changefeed frontier. This means that if the frontier writes a checkpoint before it receives updates covering the previous checkpoint from the aggregators, it could overwrite the checkpoint with less information (i.e. the checkpoint can regress). This is undesirable because we would end up losing saved progress for no good reason. An easy solution would be to just pass the checkpoint to the changefeed frontier too or we can have the frontier read it off the job progress.

Relevant code comment:

cockroach/pkg/ccl/changefeedccl/changefeed_processors.go

Lines 1280 to 1296 in 1caa3e4

	// Recover highwater information from job progress.
	// Checkpoint information from job progress will eventually be sent to the
	// changeFrontier from the changeAggregators. Note that the changeFrontier
	// may save a new checkpoint prior to receiving all spans of the
	// aggregators' frontier, potentially missing spans that were previously
	// checkpointed, so it is still possible for job progress to regress.
	p := job.Progress()
	if ts := p.GetHighWater(); ts != nil {
	cf.highWaterAtStart.Forward(*ts)
	cf.frontier.initialHighWater = *ts
	for _, span := range cf.spec.TrackedSpans {
	if _, err := cf.frontier.Forward(span, *ts); err != nil {
	cf.MoveToDraining(err)
	return
	}
	}
	}

Jira issue: CRDB-42593

Epic CRDB-42869

[blathers-crl]

cc @cockroachdb/cdc