HDD and SSD used bytes counters #4727
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
| mutable size_t LastKnownSize = 0; | ||
| }; | ||
|
|
||
| enum class EUserFacingStorageType { |
There was a problem hiding this comment.
This enum highlights that we categorize numerous storage pool kinds into just two reported storage types: HDD and SSD.
ydb/core/tx/schemeshard/schemeshard__operation_common_subdomain.h
Outdated
Show resolved
Hide resolved
jepett0
force-pushed
the
SchemeShard.per_channel_storage_limits.6
branch
from
4fca557 to
29d5c2c
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Log level needs to be TRACE to see the raw table stats received by the SchemeShard
- fix style - add descriptive comment about the nature of the storage pool kind mapping
jepett0
force-pushed
the
SchemeShard.per_channel_storage_limits.6
branch
from
29d5c2c to
2003c15
Compare
|
⚪
|
|
⚪
|
2 participants
#3275
In this PR two new counters are added:
resources.storage.used_bytes.ssdresources.storage.used_bytes.hddNecessary subcounters like
resources.storage.used_bytes.table.ssd(note the addedtablepart) are also added. These counters are needed to monitor SSD and HDD storage usage separately.In the follow-up PR
.ssdand.hddversions ofresources.storage.limit_bytesare added too.Probably the most interesting part of this PR is the "hacky" mapping of storage pool kinds to
ssdandhddcounters. It is discussed in more detail in these comments:The data for the
used_bytescounters is sourced from the SchemeShard's counters:SchemeShard/DiskSpaceTablesTotalBytesOnSsdSchemeShard/DiskSpaceTablesTotalBytesOnHddwhich are updated each time the SchemeShard receives TEvPeriodicTableStats from DataShards.
Currently, only row-oriented tables report disk space usage separately by channels, so effectively:
resources.storage.used_bytes.ssdis equal toresources.storage.table.used_bytes.ssd(note the addedtablepart)resources.storage.used_bytes.hddis equal toresources.storage.table.used_bytes.hdd(note the addedtablepart)In the future, we will start accounting for HDD and SSD usage separately for both column-oriented tables and topics. Consequently,
resources.storage.used_bytes.ssdwill more accurately represent the total SSD space used by all entities in the database.P.S. For the best reviewing experience, I recommend reviewing the commits individually.