Planned Maintenance on Cloud Cluster – Cloud software upgrades starting 31/10/22

For a 3 week period commencing 31/10/22 we will be performing essential software upgrades to the cloud software on all of the hardware nodes which make up the cloud cluster. The upgrades will require a reboot of each hardware node after the software update has completed. To minimise disruption, the work will be performed after 10 pm each night and should be completed in around 1-2 hours, but the downtime on VPS servers will be limited to just a few minutes.
To minimise downtime, prior to the node upgrade, we will migrate the VPS servers on that hardware node to an alternate node, which will cause around 1-2 minutes of downtime per server (no data transfer is required).

Following the node upgrade, the VPS servers will be migrated back to the original node, which will again cause around 1-2 minutes of downtime per server. So, in total there will be only 2 downtime events per server lasting 1-2 minutes each during the 3 weeks of work.

The hardware nodes also contain SSD storage for the cluster. There are 3 copies of all data, so when we shut down the node, the VPS servers will continue to function normally with data from the other two data copies.
During the hardware node reboot, the cluster will start replicating all of the data on the stopped hardware in order to maintain 3 copies and will spread that data around all of the other nodes. This replication of data will cause a higher than normal load on the storage cluster causing higher data access latency. This in turn may cause some higher than normal load on the VPS servers until the replication process is completed, which should take around 20 minutes.
Therefore, during this maintenance, the VPS servers should be considered at risk of higher than normal load and increased response times for a 20 minute period from around 11 pm.

The new software has been on test and in service on an existing hardware node for around 2 months, so we do not expect the new software to cause any unforeseen issues.

Services affected: VPS Servers, Shared Hosting, Hosted Email.