Our monitoring alerted us that the affected node was no longer sending data. Initial investigaton found the systems filesystem to be in read-only.
The system was then booted into a rescue environment for further investigation. From the rescue environment, the drive was found to not be discoverable by the operating system, indicating that there was a deeper issue at hand.
The data center provider was then contacted about the situation. They identified a possible issue with the drives firmware and performed a firmware update.
Following the firmware update, the drive became discoverable by the operating system again, and the system successfully booted into the main OS with passing self-tests on the drive.
