The alert is triggered when a physical disk frequently experiences I/O timeouts or aborts, or when I/O requests on the disk enter an abnormal I/O queue.
The physical disk { .labels._serial_number }({ .labels._device }) on the host { .labels.hostname } experienced I/O block and has been taken offline.
The physical disk { .labels._serial_number }({ .labels._device }) on the SCVM { .labels.hostname } experienced I/O block and has been taken offline.
The system automatically takes the physical disk with I/O blocking offline and initiates data recovery. Once taken offline, the disk stops handling any I/O requests. The offline physical disk cannot be unmounted via the AOC interface until its offline status is cleared.
Before the disk is automatically taken offline, the I/O of some volumes in the cluster may be briefly affected and become slow. Normal operation resumes once the disk is taken offline.
This issue is typically caused by firmware errors or internal media errors within the physical disk itself.
On the node where the physical disk resides, run sdm -l | less to view the logs of the sd-offline service on the node, and check the following statistics for the disk:
I/O timeout count
SCSI command unexpected abort count
I/O error handling queue processing count
When the statistics meet threshold conditions, the physical disk is marked as offline and the alert is triggered. A maximum of one physical disk can be marked as offline in the cluster at a time.
Unmount and replace the faulty disk.
After the replacement, manually resolve the alert.
