unit test fails in TestSingle

[SunRunAway]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do?
   If possible, provide a recipe for reproducing the error.

https://internal.pingcap.net/idc-jenkins/blue/organizations/jenkins/tidb_ghpr_unit_test/detail/tidb_ghpr_unit_test/14819/pipeline/

[2019-11-11T06:44:57.662Z] --- FAIL: TestSingle (0.08s)

[2019-11-11T06:44:57.662Z]     manager_test.go:97: owner 5cf3b8a5-c350-4d90-a05a-22ef71e42ea3 is not empty

2. What did you expect to see?

3. What did you see instead?

4. What version of TiDB are you using (tidb-server -V or run select tidb_version(); on TiDB)?