API Doc
Search Docs...
⌘ K
OverviewDeploymentManagementOperationReferenceGlossary
    ACOS 6.3.0
  • Acrfra Cloud Operation System cluster>
  • ACOS fault handling>
  • Snapshot plans

Consecutive failures of all snapshot plans

Description

The alert is triggered when all snapshots in the cluster fail consecutively over a period, and the number of failures exceeds the configured threshold.

Alert message

The snapshot plan has failed more than { .threshold } consecutive times.

Impact

Consecutive failures in a snapshot plan may leave the corresponding data unprotected, posing a data security risk.

Cause

This may be caused by network anomalies or abnormal restarts of the snapshot plan service, which cause the snapshot service to fail when delivering requests. Multiple retries of snapshot requests within a short period all fail, exceeding the threshold and triggering the failure alert.

Solution

  • Check the cluster's storage network connection to confirm that the storage network is normal.

  • Check whether the Zookeeper service is experiencing anomalies, and whether the Meta service is experiencing frequent Leader switches. If either of these conditions is present, it may cause the snapshot plan service to restart frequently.