Standardize the use of etcd

[JmPotato]

Background

PD owns an embedded etcd to store all kinds of things, such as:

• Cluster meta info
• Region/Store info
• TSO info
• Placement Rules info
• ...

More than these, some other components will also write things into etcd, such as:

• TiDB DDL
• TiKV GCSafePoint
• CDC task info
• TiUP/TiDB-Operator

For now, the use of etcd within PD and by these components is highly casual, which caused some problems.

• Unify the key path management #4284
• dcLocation may have the same name as an existing module and cause an accident #4109
• EtcdWorker ticks too often pingcap/tiflow#3112

So we need to standardize the use of etcd.

Development Task

To achieve this, we need to remedy these behaviors from two perspectives.

Internal

• Standardize the use of PD Internal storage, including etcd.
  • Unify the KV interface call #3453
• Provide a unified key path management to ensure we don't mess up with the etcd key space.
  • Unify the key path management #4284
  • dcLocation may have the same name as an existing module and cause an accident #4109

External

• Provide a unified external interface to read and write etcd.
• Refine etcd-related usage statistics and trace information.

[JmPotato]

Namespace Quota Feature Proposal is an RFC to add namespace isolation for multiple users in a single etcd cluster while limiting the quota for each of them. It seems that we can benefit from it also.