From our friends at Neotys
One of the most dangerous moves software developers and testers can make is being lulled into a false sense of security. For example, when application features and performance levels meet expectations during pre-production, only to crash and burn when presented to real users in production.
In that same vein, if your organization has any kind of performance testing strategy, chances are you’re conducting load testing. However, you may not be truly emulating the real world behavior of your end users in your load tests.
Realism in load tests, when overlooked, can cause a myriad of performance problems in production, and end users won’t wait around. If you’re not performing accurate and realistic load testing, you risk revenue loss, brand damage and diminished employee productivity. The solution: cloud-based load testing.
Right off the bat, the cloud provides two major advantages to load and performance procedures that help testing teams better model realistic behavior: instant infrastructure and geographic location. Cloud-based load testing also lowers the total cost of ownership, increases flexibility and allows testers to understand the impact of third-party components. So you’re sold on using the cloud; here’s what you should look for in a cloud-based load testing solution.
Make sure it includes the following must-haves:
Take into account the extent to which the tool integrates with the cloud. If you select a solution that isn’t integrated with one or more cloud platforms, you’ll need to handle several steps manually—like learning how each platform you’ll be using works (including its limitations & constraints) and building, maintaining and accessing the machines you want to use.
Your cloud-based load testing solution should offer integration with the cloud to simplify and accelerate the steps needed to use the cloud infrastructure. Your chosen tool should also enable fast provisioning, simplified security, improved scalability, a unified interface, advanced test launching and advanced results reporting.
While cloud-based load testing is, in many cases, more realistic than testing in the lab, simply moving to the cloud isn’t enough to ensure the most realistic tests. To truly add a layer of realism to your cloud-based tests, you must account for bandwidth constraints experienced by real users.
Your cloud-based load testing tool should:
Cloud-based load testing is advantageous, to say the least. However, it shouldn’t take the place of lab testing. Rather, the two should be used in tandem to test from inside/outside of the firewall and to validate application performance for any user regardless of device, location, network and/or bandwidth constraints.
Look for a single solution that will allow your organization to reuse scripts for both lab testing and cloud testing. This type of tool will save you time and effort, while enabling you to pinpoint performance problems that show up in cloud load testing but not in internal tests. Ultimately, a unified solution will lower licensing/training costs and allow test engineers to use their existing skills for both types of load testing.
Creating and running your cloud-based load tests is only one part of the equation. After the tests have run, you’ll need to understand how your application behaved under load. Analysis tools are required to ensure the app met predetermined performance requirements.
With your cloud-based load testing tool, you should be able to easily analyze the collected data and create actionable reports that describe the performance of the tested application. Make sure your tool also includes a comprehensive monitoring system—understanding what was happening within the infrastructure of your application during the load test is critical in identifying any root issues.
This one is pretty straightforward. Whether you’re using Adobe Flex, Microsoft Silverlight, Real-Time Messaging Protocol (RTMP), AJAX push technologies, WebSockets, etc., you’ll need a tool with built-in support for tech you’re using. Without this support, effectively testing your application’s performance is near impossible.
The cloud provides organizations with the opportunity to improve the scale and realism of load testing, while saving time and lowering overall costs. To meet user expectations in terms of application performance, cloud-based load testing is a must! Just remember, simply moving to the cloud isn’t enough—a primary driver of your success will be the tool you use and how well it uses cloud technology. Choose wisely!