Affected
Partial outage from 4:09 PM to 10:18 PM
- ResolvedResolved
System back online
This backup system is now back online and accessible again.
Some users who were performing actions on their backups when the system went down may have their backup status stuck. If this applies to you, please open a ticket and our team will get this corrected.
Rundown of the issue
At around 1PM BST, our monitoring systems flagged unusually high I/O wait and CPU usage on this backup node. As the issue did not resolve on its own, our team began investigating possible faults on the system.
While this investigation was underway, all new backups were redirected to another backup server to prevent new backup jobs from being affected.
The high I/O wait and CPU usage continued, but there were initially no clear error logs showing the exact cause. During the investigation, the node crashed, which left the operating system in a bad state and prevented it from booting normally.
Our team then booted the server into rescue mode and performed a boot repair to get the operating system back into a working state.
The system has now booted successfully and is accessible again.
We’ll continue to monitor the node closely and investigate the underlying storage issue that caused the original I/O problems. New backups until we're sure the system is fully stable will be directed to another backup system. But all backups are now accessible again.
- InvestigatingInvestigating
A hardware error has been flagged on one of our backup systems, causing it to become unresponsive.
All new backups have been diverted to another BORG system while we investigate the error. However, if you have a backup already stored on this system, it may be inaccessible while we work on resolving the issue.
We are working to get this system accessible again as quickly as possible.

