Skip to content

Commit

Permalink
Merge branch 'master' into update_set_tikv_gc_life_time
Browse files Browse the repository at this point in the history
  • Loading branch information
@csuzhangxc
csuzhangxc authored Apr 28, 2021
2 parents 447afdb + 6f1f091 commit 717c6ee
Show file tree
Hide file tree
Showing 21 changed files with 1,402 additions and 216 deletions.
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ endif

ALL_CHECKS = EOF codegen boilerplate openapi-spec crd-groups spelling

check: $(addprefix check-,$(ALL_CHECKS)) lint tidy
check: $(addprefix check-,$(ALL_CHECKS)) lint tidy

check-%:
./hack/verify-$*.sh
Expand Down
129 changes: 129 additions & 0 deletions docs/api-references/docs.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -852,6 +852,23 @@ Kubernetes core/v1.PodSecurityContext
<p>PodSecurityContext of the component</p>
</td>
</tr>
<tr>
<td>
<code>topologySpreadConstraints</code></br>
<em>
<a href="#topologyspreadconstraint">
[]TopologySpreadConstraint
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>TopologySpreadConstraints describes how a group of pods ought to spread across topology
domains. Scheduler will schedule pods in a way which abides by the constraints.
This field is is only honored by clusters that enables the EvenPodsSpread feature.
All topologySpreadConstraints are ANDed.</p>
</td>
</tr>
</table>
</td>
</tr>
Expand Down Expand Up @@ -1663,6 +1680,23 @@ Kubernetes core/v1.PodSecurityContext
<p>PodSecurityContext of the component</p>
</td>
</tr>
<tr>
<td>
<code>topologySpreadConstraints</code></br>
<em>
<a href="#topologyspreadconstraint">
[]TopologySpreadConstraint
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>TopologySpreadConstraints describes how a group of pods ought to spread across topology
domains. Scheduler will schedule pods in a way which abides by the constraints.
This field is is only honored by clusters that enables the EvenPodsSpread feature.
All topologySpreadConstraints are ANDed.</p>
</td>
</tr>
</table>
</td>
</tr>
Expand Down Expand Up @@ -3769,6 +3803,10 @@ FlashSecurity
</tr>
</tbody>
</table>
<h3 id="component">Component</h3>
<p>
<p>Component defines component identity of all components</p>
</p>
<h3 id="componentaccessor">ComponentAccessor</h3>
<p>
<p>ComponentAccessor is the interface to access component details, which respects the cluster-level properties
Expand Down Expand Up @@ -4090,6 +4128,23 @@ employed to update Pods in the StatefulSet when a revision is made to
Template.</p>
</td>
</tr>
<tr>
<td>
<code>topologySpreadConstraints</code></br>
<em>
<a href="#topologyspreadconstraint">
[]TopologySpreadConstraint
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>TopologySpreadConstraints describes how a group of pods ought to spread across topology
domains. Scheduler will schedule pods in a way which abides by the constraints.
This field is is only honored by clusters that enables the EvenPodsSpread feature.
All topologySpreadConstraints are ANDed.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="configmapref">ConfigMapRef</h3>
Expand Down Expand Up @@ -4776,6 +4831,23 @@ Kubernetes core/v1.PodSecurityContext
<p>PodSecurityContext of the component</p>
</td>
</tr>
<tr>
<td>
<code>topologySpreadConstraints</code></br>
<em>
<a href="#topologyspreadconstraint">
[]TopologySpreadConstraint
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>TopologySpreadConstraints describes how a group of pods ought to spread across topology
domains. Scheduler will schedule pods in a way which abides by the constraints.
This field is is only honored by clusters that enables the EvenPodsSpread feature.
All topologySpreadConstraints are ANDed.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="dmclusterstatus">DMClusterStatus</h3>
Expand Down Expand Up @@ -19932,6 +20004,23 @@ Kubernetes core/v1.PodSecurityContext
<p>PodSecurityContext of the component</p>
</td>
</tr>
<tr>
<td>
<code>topologySpreadConstraints</code></br>
<em>
<a href="#topologyspreadconstraint">
[]TopologySpreadConstraint
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>TopologySpreadConstraints describes how a group of pods ought to spread across topology
domains. Scheduler will schedule pods in a way which abides by the constraints.
This field is is only honored by clusters that enables the EvenPodsSpread feature.
All topologySpreadConstraints are ANDed.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="tidbclusterstatus">TidbClusterStatus</h3>
Expand Down Expand Up @@ -20774,6 +20863,46 @@ BasicAutoScalerStatus
</tr>
</tbody>
</table>
<h3 id="topologyspreadconstraint">TopologySpreadConstraint</h3>
<p>
(<em>Appears on:</em>
<a href="#componentspec">ComponentSpec</a>,
<a href="#dmclusterspec">DMClusterSpec</a>,
<a href="#tidbclusterspec">TidbClusterSpec</a>)
</p>
<p>
<p>TopologySpreadConstraint specifies how to spread matching pods among the given topology.
It is a minimal version of corev1.TopologySpreadConstraint to avoid to add too many fields of API
Refer to <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints">https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints</a></p>
</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>topologyKey</code></br>
<em>
string
</em>
</td>
<td>
<p>TopologyKey is the key of node labels. Nodes that have a label with this key
and identical values are considered to be in the same topology.
We consider each <key, value> as a &ldquo;bucket&rdquo;, and try to put balanced number
of pods into each bucket.
MaxSkew is default set to 1
WhenUnsatisfiable is default set to DoNotSchedule
LabelSelector is generated by component type
See pkg/apis/pingcap/v1alpha1/tidbcluster_component.go#TopologySpreadConstraints()</p>
</td>
</tr>
</tbody>
</table>
<h3 id="txnlocallatches">TxnLocalLatches</h3>
<p>
(<em>Appears on:</em>
Expand Down
33 changes: 33 additions & 0 deletions examples/advanced/tidb-cluster.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,12 @@ spec:
## Set this in the case where a TiDB cluster is deployed across multiple Kubernetes clusters. default to empty.
# clusterDomain: cluster.local

## TopologySpreadConstraints for pod scheduling, will be overwritten by each cluster component's specific spread constraints setting
## Can refer to PD/TiDB/TiKV/TiCDC/TiFlash/Pump topologySpreadConstraints settings, and ensure only cluster-scope general settings here
## Ref: pkg/apis/pingcap/v1alpha1/types.go#TopologySpreadConstraint
# topologySpreadConstraints:
# - topologyKey: topology.kubernetes.io/zone

###########################
# TiDB Cluster Components #
###########################
Expand Down Expand Up @@ -315,6 +321,11 @@ spec:
## Ref: https://docs.pingcap.com/tidb-in-kubernetes/stable/enable-tls-for-mysql-client
# tlsClientSecretName: custom-tidb-client-secret-name

## TopologySpreadConstraints for pod scheduling, will overwrite cluster level spread constraints setting
## Ref: pkg/apis/pingcap/v1alpha1/types.go#TopologySpreadConstraint
# topologySpreadConstraints:
# - topologyKey: topology.kubernetes.io/zone

tidb:
############################
# Basic TiDB Configuration #
Expand Down Expand Up @@ -528,6 +539,11 @@ spec:
# - tidb
# topologyKey: kubernetes.io/hostname

## TopologySpreadConstraints for pod scheduling, will overwrite cluster level spread constraints setting
## Ref: pkg/apis/pingcap/v1alpha1/types.go#TopologySpreadConstraint
# topologySpreadConstraints:
# - topologyKey: topology.kubernetes.io/zone

tikv:
############################
# Basic TiKV Configuration #
Expand Down Expand Up @@ -721,6 +737,11 @@ spec:
# - tikv
# topologyKey: kubernetes.io/hostname

## TopologySpreadConstraints for pod scheduling, will overwrite cluster level spread constraints setting
## Ref: pkg/apis/pingcap/v1alpha1/types.go#TopologySpreadConstraint
# topologySpreadConstraints:
# - topologyKey: topology.kubernetes.io/zone

## Deploy TiDB Binlog of a TiDB cluster
## Ref: https://pingcap.com/docs/tidb-in-kubernetes/stable/deploy-tidb-binlog/#deploy-pump
# pump:
Expand Down Expand Up @@ -757,6 +778,10 @@ spec:
# # Ref: https://docs.pingcap.com/tidb/stable/tidb-binlog-configuration-file#pump
# config: |
# gc = 7
# # TopologySpreadConstraints for pod scheduling, will overwrite cluster level spread constraints setting
# # Ref: pkg/apis/pingcap/v1alpha1/types.go#TopologySpreadConstraint
# topologySpreadConstraints:
# - topologyKey: topology.kubernetes.io/zone

## TiCDC is a tool for replicating the incremental data of TiDB
## Ref: https://pingcap.com/docs/tidb-in-kubernetes/stable/deploy-ticdc/
Expand Down Expand Up @@ -796,6 +821,10 @@ spec:
# gcTTL: 86400
# logLevel: info
# logFile: /dev/stderr
# # TopologySpreadConstraints for pod scheduling, will overwrite cluster level spread constraints setting
# # Ref: pkg/apis/pingcap/v1alpha1/types.go#TopologySpreadConstraint
# topologySpreadConstraints:
# - topologyKey: topology.kubernetes.io/zone

## TiFlash is the columnar storage extension of TiKV
## Ref: https://pingcap.com/docs/tidb-in-kubernetes/stable/deploy-tiflash/
Expand Down Expand Up @@ -869,3 +898,7 @@ spec:
# proxy: |
# [security]
# cert-allowed-cn = CNNAME
# # TopologySpreadConstraints for pod scheduling, will overwrite the cluster level spread constraints setting
# # Ref: pkg/apis/pingcap/v1alpha1/types.go#TopologySpreadConstraint
# topologySpreadConstraints:
# - topologyKey: topology.kubernetes.io/zone
30 changes: 30 additions & 0 deletions manifests/crd.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2433,6 +2433,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
required:
Expand Down Expand Up @@ -4517,6 +4520,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
required:
Expand Down Expand Up @@ -6565,6 +6571,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
required:
Expand Down Expand Up @@ -8754,6 +8763,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
required:
Expand Down Expand Up @@ -10807,6 +10819,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
required:
Expand Down Expand Up @@ -12868,6 +12883,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
required:
Expand All @@ -12892,6 +12910,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
type: object
Expand Down Expand Up @@ -15283,6 +15304,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
required:
Expand Down Expand Up @@ -15372,6 +15396,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
worker:
Expand Down Expand Up @@ -17424,6 +17451,9 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
items: {}
type: array
version:
type: string
required:
Expand Down
10 changes: 9 additions & 1 deletion pkg/apis/pingcap/v1alpha1/dmcluster_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ func TestDMMasterIsAvailable(t *testing.T) {
}
}

// TODO: refector test of buildDMClusterComponentAccessor
func TestDMComponentAccessor(t *testing.T) {
g := NewGomegaWithT(t)

Expand All @@ -110,7 +111,14 @@ func TestDMComponentAccessor(t *testing.T) {
testFn := func(test *testcase, t *testing.T) {
t.Log(test.name)

accessor := buildDMClusterComponentAccessor(test.cluster, test.component)
dc := &DMCluster{
ObjectMeta: metav1.ObjectMeta{
Name: "test",
},
Spec: *test.cluster,
}

accessor := buildDMClusterComponentAccessor(ComponentDMMaster, dc, test.component)
test.expectFn(g, accessor)
}
affinity := &corev1.Affinity{
Expand Down
Loading

0 comments on commit 717c6ee

Please sign in to comment.