Slow website loading times
Incident Report for ShopWired
Postmortem

Throughout April 2021, ShopWired engineers had been carrying out a number of upgrades across our platform infrastructure involving multiple servers and the software used by them as well as other third party services in use.

These upgrades were completed on 29th April 2021.

At around 5.57pm on 4th May 2021 our monitoring software alerted us that the platform was experiencing difficulties in serving traffic. The error became apparent following a large increase in traffic to a few websites utilising ShopWired that had product launches.

During this time, some visitors to websites on the ShopWired platform will have been shown a 503 error which is an error message shown by our load balancer software when it cannot direct traffic to a web server.

By around 6.30pm traffic returned to normal levels and the platform began to be able to serve all visitors normally.

ShopWired engineers began investigating the errors immediately to establish the root cause.

A large increase in traffic to a single website on ShopWired's network was then received on 5th May that started at around 5.56pm.

As before, some visitors to websites on the ShopWired platform will have seen a 503 error.

This problem lasted until around 8.45pm on 5th May at which time traffic levels returned to normal.

ShopWired engineers continued to work on the problem on 6th and 7th May and implemented final fixes that rectified the underlying cause on the 7th May, identified as a software failure.

The issue was difficult to identify and unfortunately only presented itself at times of peak traffic onto the platform.

We have carried out load tests on the infrastructure to ensure that it is able to handle peak demand normally.

ShopWired engineers will now conduct more thorough load testing after deployments to ensure reliability.

Posted May 19, 2021 - 22:56 BST

Resolved
ShopWired websites and admin are now operating normally and this issue is being marked as resolved. We are continuing to investigate the underlying causes with the affected parts of our infrastructure to remedy issues for future performance.
Posted May 06, 2021 - 00:04 BST
Monitoring
The problem has resolved and we are currently monitoring performance.
Posted May 05, 2021 - 20:35 BST
Investigating
We are experiencing slow site loading times and occasional 503 errors across websites and when using ShopWired accounts that appears to be the result of the same infrastructure issue that affected users yesterday. We are working to restore service as soon as possible.
Posted May 05, 2021 - 18:53 BST
This incident affected: Websites and ShopWired Admin System.