Skip to content

gcTS and alertGCTS aren't meaningful #1262

Open
@AricSu

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?
    Because of refine gc strategy of pump #646
    We can see that a formula alertGCTS := gcTS + int64(storage.EncodeTSO(alertGCMS)) adds extra 20h to avoid the scenarios of disk space of the pump is full and much impact on core system formation.
    But, the metric and error log aren't accurate and easy understand to user.
drainer's checkpoint is older than pump gc ts, some binlogs are purged
  1. What did you expect to see?

something likes this one

after 20h, drainer's checkpoint will be older than pump gc ts, some binlogs will be purged, please handle it manually based on gc ts as soon as possible
  1. What did you see instead?
drainer's checkpoint is older than pump gc ts, some binlogs are purged

and gcTs == alertGCTS --> https://github.com/pingcap/tidb-binlog/pull/646/files#diff-d73c6b7c1637ab38367029c88ed0bf596762627453dab635429f991ae9dfc9acR589-R596

		if drainer.MaxCommitTS < gcTS {
			log.Error("drainer's checkpoint is older than pump gc ts, some binlogs are purged",
				zap.String("drainer", drainer.NodeID),
				zap.Int64("gc ts", gcTS),
				zap.Int64("drainer checkpoint", drainer.MaxCommitTS),
			)
			// will add test when binlog have failpoint
			detectedDrainerBinlogPurged.WithLabelValues(drainer.NodeID).Inc()
		}
	}
}
  1. Please provide the relate downstream type and version of drainer.
    (run drainer -V in terminal to get drainer's version)

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions