lightning: replace precheck KvAvailable/KvCapacity < 10% with KvAvailable/KvUsedSize < 10%

[cyliu0]

Feature Request

Is your feature request related to a problem? Please describe:

Currently the lightning precheck "KvAvailable/KvCapacity < 10%" will lead to precheck failure when the tikv disks have some other data.

Describe the feature you'd like:

Replace the lightning precheck item KvAvailable/KvCapacity < 10% with KvAvailable/KvUsedSize < 10%. This seems to be more reasonable.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

[cyliu0]

@IANTHEREAL PTAL

[IANTHEREAL]

The worst case is that local import will fail, but you can skip the check. But if we change, what's the worst case? Would it destroy the TiKV cluster? @cyliu0

[cyliu0]

The worst case is that local import will fail, but you can skip the check. But if we change, what's the worst case? Would it destroy the TiKV cluster? @cyliu0

No, I don't think so. Because I believe the current check item is not reasonable since the KvCapacity represents the whole disk/partition which the kv dir located. For example, if we have a 1TB disk, the kv dir is 1GB and the left space is 100GB, we can't import anything even if it's hundreds of KB with lightning based on the current check item. I do believe kv should be working properly in this scenario

[IANTHEREAL]

I know your example, but you don't seem to understand my point. I mean, are there cases worse after this modification?

what's prechck KvAvailable/KvCapacity used for? To protect TiKV from being overwritten? @cyliu0 if we have a 1TB disk, and the left space is 100GB, but we want to import 99 GB kv pairs, which check can prevent this from happening

[cyliu0]

I know your example, but you don't seem to understand my point. I mean, are there cases worse after this modification?

what's prechck KvAvailable/KvCapacity used for? To protect TiKV from being overwritten? @cyliu0 if we have a 1TB disk, and the left space is 100GB, but we want to import 99 GB kv pairs, which check can prevent this from happening

No, it can't. A better version should be (KvAvailable - EstimateSizePerKV) / (KvUsedSize + EstimateSizePerKV) < 10%

[IANTHEREAL]

yes, I think it's better @Little-Wallace what about you?

[Little-Wallace]

We do not want to import data to TiKV until disk is full. It will make tikv can not start again.

[IANTHEREAL]

The purpose is the same, can you describe how to do now, such as the example I gave above, how to prevent it?

[Little-Wallace]

The used of TiKV disk can not be be accurately estimated before they are imported to TiKV.

[cyliu0]

So this is not estimated size? @Little-Wallace @IANTHEREAL

| 7 | Cluster available is rich, available is 3.173TiB, we need 3.588GiB                             | critical    | true   |

[Little-Wallace]

@cyliu0 what we need is just an estimated size. It is not true.