One of our clients recently set us the challenge of hitting some significant thresholds for concurrent users against their website. The website hadn’t yet gone live so we had the luxury of being able to hit the backend infrastructure with everything we have, but we quickly realised that instead of being limited by the configuration or sheer scale of the back end infrastructure we had set up, we were being limited by the elastic load balancer itself. Although much of the ELB is a black box which we only have limited visibility of through Cloudwatch metrics and through the configuration options available through the AWS console we do know that part of the functionality of the ELB is it’s ability to scale up through the various AWS instances. During testing using large loads, initially targeting 300,000 concurrent users writing once per minute to a back end PostgreSQL database, quickly demonstrated that the ELB is vulnerable to large spikes in traffic and we found that during its scaling activities significant numbers of dropped connections were occurring. Without knowledge of exactly when a ‘scaling event’ is triggered or what the trigger for the scale is, we cannot reliably predict the traffic profile through a large volume load test. Fortunately AWS have come to the rescue by way of the ELB pre-warming service. Only available for AWS accounts signed up to $49 per month developer support service, pre-warming allows you to specify the type, duration and volume of load expected on your ELB, AWS will then scale the ELB accordingly so it is ready for testing – or for any expected real world load. We found the service easy to request, coupled with AWS’s usual responsive support we were able to ready an ELB in less than an hour and our tests no longer suffered from the annoying saw tooth profile we had seen as the base ELB traversed it’s scaling events.
It’s evident from these cloudwatch graphs that pre-warming ELB goes a long way to smoothing out the ELB’s ability to handle large spikes in load.
Cloudten Industries © is an Australian cloud practice and a recognised consulting partner of AWS. We specialise in the design, delivery and support of cloud based solutions. Don’t hesitate to contact us if you have any queries about this post or any cloud related topic.