API Doc
Search Docs...
⌘ K
OverviewDeploymentManagementOperationReferenceGlossary
    ACOS 6.3.0
  • Acrfra Cloud Operation System cluster>
  • ACOS fault handling>
  • Physical disks

Unhealthy physical disk

Description

The alert is triggered when a physical disk experiences read or write errors, or when read or write requests time out or receive no response, indicating that the disk has entered an unhealthy state.

Alert message

  • Host { .labels.hostname }: The physical disk { .labels._serial_number }({ .labels._device }) is unhealthy. The system will automatically isolate the disk and restore the data to other healthy physical disks. Do not pull the disk.

  • Host { .labels.hostname }: The physical disk { .labels._serial_number }({ .labels._device }) failure and is now unmounted and can be removed safely.

  • Host { .labels.hostname }: Disk { .labels._serial_number }({ .labels._device }) failure and has been isolated. Please unmount the remaining partitions on it.

  • SCVM { .labels.hostname }: Disk { .labels._serial_number }({ .labels._device }) failure and has been isolated. Please unmount the remaining partitions on it.

  • SCVM { .labels.hostname }: The physical disk { .labels._serial_number }({ .labels._device }) is unhealthy. The system will automatically isolate the disk and restore the data to other healthy physical disks. Do not pull the disk.

  • SCVM { .labels.hostname }: The physical disk { .labels._serial_number }({ .labels._device }) failure and is now unmounted and can be removed safely.

Impact

The system automatically isolates the unhealthy disk and migrates the data on it to other healthy physical disks. After isolation is complete, the storage capacity provided by the faulty disk is counted as failed space.

Cause

The following causes trigger an unhealthy disk alert. After the system completes data recovery or migration, the physical disk will no longer be in use. The alert will change to indicate that the disk has failed and is now in an unmounted state, and can be safely removed.

CauseInformation to check
I/O timeout or abort causes the zbs-chunk service to trigger errflag.On the node where the disk resides, run zbs-node show_disk_status /dev/sdxx to check the I/O error count and disk status flags for the disk. Run less /var/log/messages to view the messages log.
I/O errors (accumulated more than 300) or excessive data verification errors (accumulated more than 100) cause the zbs-chunk service to trigger errflag.On the node where the disk resides, run zbs-node show_disk_status /dev/sdxx to check the I/O error count and disk status flags for the disk. Run less /var/log/messages to view the messages log.

Solution

  1. Unmount and replace the faulty disk.

  2. After the replacement, manually resolve the alert.