From our partners at Neotys
By now, hopefully you’re well aware of the importance of mobile performance testing. It’s essential for your organization to test for mobile users whether or not you have a dedicated mobile application. In case you aren’t aware of its necessity, here are a couple statistics to cement the idea:
To conduct complete mobile performance testing, a process that will allow you to consistently deliver an exceptional application experience to mobile users, make sure you hit every point on our ultimate mobile performance testing checklist:
If you ignore the needs of the business, you risk testing the wrong aspects of your application. Identify which aspects of the app experience drive meaningful metrics and then determine the user experience requirements relative to these metrics.
Your key performance indicators (KPIs) will depend on your business objectives. Clarify your goals early on and consider these important KPIs when initiating mobile performance testing: average requests per second, average request response time, maximum response time, average throughput, and error rate.
Building a mobile performance testing suite without any knowledge of how the code works may cause you to overlook significant problems and bottlenecks. Work with your developers to identify parts of the application that should be tested based on the way the application is built. Also, design your tests in conjunction with Dev to gain a comprehensive understanding of the application areas that need to be tested.
Executing all of your tests across every combination of variants is both time and cost intensive. Initially, you’ll want to use your KPIs and behavioral analytics to determine the 10-20 combinations you wish to run in the first round of testing. By focusing on these selected combinations, you’ll have a better idea of the types of scenarios your application can and cannot handle. Obviously, you should fix any bottleneck or bug before moving on to the next mobile performance testing scenario.
Mobile traffic spikes can affect everyone on your application. Consult your analytics platform to ensure your load testsaccurately reflect realistic proportions of mobile users and web users—this is especially important if your mobile and web sites share a backend.
To make the testing environment and user experience as realistic as possible, utilize emulators. These allow you to test various phones, tablets, etc. without having to physically own and/or manage those devices. While emulating devices, be sure to take into account parallel connections as mobile networks, devices and applications all enforce varying policies in terms of how many are allowed at a given time. You’ll also need to customize the user agent header in your scripts to ensure the web or mobile server is sending the correct content to all of your users’ different devices.
Mobile devices will access your servers over 3rd-party carrier networks that vary greatly in speed, latency, bandwidth, etc. By employing network virtualization, you can mimic the network characteristics that mobile devices actually experience. Prioritize the characteristics you see most often, but don’t ignore how your application performs for users on the extreme ends of the spectrum. You’ll want to make sure that every mobile user’s experience is the best it can be, given their specific network limitations.
As you well know, your mobile users won’t be accessing your application from the same place. Depending on their location, mobile users will experience different levels of performance, and as a tester, this is something you’ll need to account for. Create a geographic profile of your users based on historical data and run geographically-dispersed load tests from the cloud.
It can be easy to overlook the importance of recording from real devices when conducting mobile performance testing, especially when you’re testing a web app that can be recorded from a browser. The problem comes when any call from the mobile interface is different from a call from the web app, or if there is not a corresponding web app. Recording your test scenarios directly from a mobile device will give you the most accurate test results in either situation.
Quickly identify the root cause of mobile performance issues by engaging in end-user experience testing on real devices. Load testing paired with functional testing on a real device will allow you to measure rendering times on the front end of your application that you can then correlate with backend times.
No matter how closely your test environment resembles your production environment, it will never be 100% the same. Sometimes bugs will only reveal themselves in the real world, under actual operation. As such, you’ll need to test for mobile users in your real-world production environment. While testing in production poses some risk, it can be riskier not to invest in the practice as you’ll end up blind to the levels of performance actually experienced by your mobile users.
People are becoming more dependent on smartphones and other connected mobile devices for quick access to applications and websites. If you aren’t testing for mobile users, you will fall short in terms of application performance.
If you complete this mobile performance testing checklist, however, you will not only appease managers and stakeholders, but you will also be able to successfully deliver exceptional application performance to all of your mobile users, regardless of device, network conditions and geographic location.