The cluster's topology awareness distributes data replicas across different topology elements (chassis, racks, and availability zones) to improve disaster recovery. However, when the resource distribution across different topology elements in the cluster is severely imbalanced such that the storage capacity of a single node exceeds the combined capacity of all other nodes, a failure of that node may leave the remaining nodes unable to recover all data to the expected security level.
If the entire { .labels.zone_name } fails, the memory in { .labels.reserved_zone_names } is insufficient to support all running or suspended VMs.
The difference between availability zones is greater than { .threshold }% of the total data space of the cluster.
If the other AZ fails, the data space in { .labels.name } is insufficient for such a recovery that all the data have { .threshold } replicas.
Rack { .labels.name }: The brick { .labels.brick } has taken more than { .threshold } of the total data space of the current rack.
Rack { .labels.rack }: This rack has taken more than { .threshold } of the total data space of the current cluster.
If any of the listed hosts fails, the remaining cluster space is inadequate to recover data to the desired redundancy level. Host: { .labels.chunks }.
When the topology element involved in the alert (host, chassis, rack, or availability zone) fails or becomes unavailable, there is a risk that some data may not be fully recoverable, and some data may become inaccessible.
For example, when the total storage capacity of a single rack significantly exceeds the combined capacity of all other racks, the system may concentrate multiple replicas across different hosts within that rack, even though it avoids placing multiple replicas on the same host. If that rack fails, the corresponding data will be inaccessible until the rack resumes normal operation.
The resource capacity (storage space, memory, or CPU cores) provided by a topology element significantly exceeds that of other topology elements at the same level, resulting in an imbalanced resource distribution.
It is recommended to optimize resource distribution, including but not limited to the following approaches:
Adjust the physical deployment of nodes to achieve more balanced resources across availability zones, racks, and chassis.
Expand the capacity of lower-spec nodes (such as adding memory or physical disks) to improve overall balance.
