The alert is triggered when the CPU or memory utilization of a host exceeds the configured threshold.
In particular, in an active-active cluster, the witness node is only responsible for the leader election and metadata backup functions. Their CPU and memory consumption is expected to remain consistently low. Any high resource utilization on the witness node may indicate an anomaly.
Host { .labels.hostname }: CPU usage has been above { .threshold } for 5 minutes.
Host { .labels.hostname }: Memory usage has been above { .threshold } for 5 minutes.
Witness node { .labels.hostname }: CPU usage has been above { .threshold } for 5 minutes.
Witness node { .labels.hostname }: Memory usage has been above { .threshold } for 5 minutes.
SCVM { .labels.hostname }: CPU usage has been above { .threshold } for 5 minutes.
SCVM { .labels.hostname }: Memory usage has been above { .threshold } for 5 minutes.
When service resource consumption exceeds the threshold, the system may automatically restart the affected service. If the service quickly returns to a high resource utilization state after restarting, repeated restarts may occur, affecting cluster stability. Assess the specific impact by referring to the description of the corresponding service anomaly.
Common causes include:
Network anomaly
When network failures occur on a node or across the cluster, a large volume of requests cannot be processed in time, causing request backlogs in affected services and abnormal increases in memory usage.
Abnormal storage configuration
Excessive memory usage in a storage service may result from physical disks being mounted via the command line beyond the node's preset specifications, causing the memory used for data indexing to exceed expected levels.
If a service resource utilization alert is accompanied by a network alert, the resource anomaly typically resolves automatically after the network issue is fixed. No additional action is required.
If storage service memory utilization is abnormal, and you have confirmed that the mounted cache disk or data disk capacity exceeds the node's default specifications, contact an Arcfra after-sales engineer to adjust the memory quota.
In other scenarios, it is recommended to collect logs and contact an Arcfra after-sales engineer directly for assistance.
