Kinetic Hosting - Notice history

There was a power failure in the data center where these systems are located. Power was restored, and after some further recovery, the nodes are now back online.

Game panel speed restored.

The following nodes are undergoing electrical maintenance, which may lead to an unavailability of the public network access, causing issues with controlling or connecting to servers located on the impacted nodes.

AUS-P11-A1-OV
AUS-P12-A1-OV
AUS-P16-A4-OV
AUS-P17-A4-OV

This incident has been resolved. The data center provider has been able to successfully acquire an emergency connection while the original connection outage is investigated.

Once the original connection is restored it will be reinstated without any expected interruption.

System back online

This backup system is now back online and accessible again.

Some users who were performing actions on their backups when the system went down may have their backup status stuck. If this applies to you, please open a ticket and our team will get this corrected.

Rundown of the issue

At around 1PM BST, our monitoring systems flagged unusually high I/O wait and CPU usage on this backup node. As the issue did not resolve on its own, our team began investigating possible faults on the system.

While this investigation was underway, all new backups were redirected to another backup server to prevent new backup jobs from being affected.

The high I/O wait and CPU usage continued, but there were initially no clear error logs showing the exact cause. During the investigation, the node crashed, which left the operating system in a bad state and prevented it from booting normally.

Our team then booted the server into rescue mode and performed a boot repair to get the operating system back into a working state.

The system has now booted successfully and is accessible again.

We’ll continue to monitor the node closely and investigate the underlying storage issue that caused the original I/O problems. New backups until we're sure the system is fully stable will be directed to another backup system. But all backups are now accessible again.

The network unavailability has been resolved by the data center provider.