Throughout April 2021, ShopWired engineers had been carrying out a number of upgrades across our platform infrastructure involving multiple servers and the software used by them as well as other third party services in use.
These upgrades were completed on 29th April 2021.
At around 5.57pm on 4th May 2021 our monitoring software alerted us that the platform was experiencing difficulties in serving traffic. The error became apparent following a large increase in traffic to a few websites utilising ShopWired that had product launches.
During this time, some visitors to websites on the ShopWired platform will have been shown a 503 error which is an error message shown by our load balancer software when it cannot direct traffic to a web server.
By around 6.30pm traffic returned to normal levels and the platform began to be able to serve all visitors normally.
ShopWired engineers began investigating the errors immediately to establish the root cause.
A large increase in traffic to a single website on ShopWired's network was then received on 5th May that started at around 5.56pm.
As before, some visitors to websites on the ShopWired platform will have seen a 503 error.
This problem lasted until around 8.45pm on 5th May at which time traffic levels returned to normal.
ShopWired engineers continued to work on the problem on 6th and 7th May and implemented final fixes that rectified the underlying cause on the 7th May, identified as a software failure.
The issue was difficult to identify and unfortunately only presented itself at times of peak traffic onto the platform.
We have carried out load tests on the infrastructure to ensure that it is able to handle peak demand normally.
ShopWired engineers will now conduct more thorough load testing after deployments to ensure reliability.