The 10 Top Best Practices of Mobile Testing

There are many things to consider when testing on a mobile platform, and it can be somewhat overwhelming, especially if you are new to mobile platform testing.  Mobile application testing is the same as testing Web applications, just on a smaller form factor, right?  Wrong.  While it is true that some basics of testing best practices apply, there are many nuances to consider that are specific to the mobile platform.  The following list is not all-inclusive; instead, it is meant to provide some fundamental guidelines to follow when testing on a mobile platform.

 

1.Ensure the test objectives align with the corporate/business requirements.  Ask the following questions:

•For functionality, how important is it that the content be correct?

•How should content be displayed on different form factors if the device is smaller or larger than the baseline design?  What is the expected result if content is cut off or if there is too much whitespace?

•How should the content display in portrait mode vs. landscape?

•How should the application behave depending on the connection type?  For example, Wi-Fi vs. cellular

•What are the requirements for the user experience?  Is performance important?  For example, the amount of time it takes to log in.

 

2.Don’t test everything.  When you take into consideration device type, operating system, carrier, and so on, there are thousands of permutations. Keep in mind that you don’t want to test all of them as initial testing plus subsequent regression tests across every combination is both time and cost intensive. Instead, use analytics to determine the top 10-20 combinations, and then set up your test environment to align with what the end-users use most often.

 

3.Build, configure, and maintain a proper mobile test lab.  If you are restricting your testing to manual only, you might consider a cloud provider to limit expenses.  However, if you are pursuing automation or have security concerns about exposing your QA environment to a cloud, an on-premise lab might be a better option.  As long as you limit your focus to the top device combinations that your customers actually use, managing and maintaining an on-premise lab is relatively easy.

 

4.Remember to plan for integration testing.  If your desktop Web site connects to the same backend as your mobile site, you will need to create test cases that perform functions on both.  For example, consider  the following test case:

•Log in using Internet Explorer, add an item to your cart on a shopping site.

•Access that same site on an iPhone and confirm the items shown in your cart matches. those placed there using your Internet Explorer browser.

•Place an order for the item using the iPhone browser.

•Log in using Internet Explorer and confirm the order was placed correctly.

 

5.Don’t be tempted to use emulators or simulators.  While they may seem like a good way to cut lab expenses, it has been proven that defects are missed and consistently leak into production when testing using emulators or simulators.  All system testing must be done on real devices to ensure proper test integrity.  After all, your customers are not going to use an emulator when they use your app, are they?

 

6.Plan for rapid change.  The release cycles for mobile applications take an average of one to four weeks, so plan to optimize your resources to deal with shortened timelines.

 

7.Use remote control tools.  There are tools available that enable you to view and control real mobile devices, similar to the way Remote Desktop works for PCs, making life a bit easier for testers.  Some of these tools are free while others carry a small fee, depending on the target platform.  Remote control tools are especially useful for today’s touchscreen smartphones, allowing use of a mouse in place of a tester’s finger to perform gestures (taps, swipes, etc.) and a desktop keyboard in place of the device’s virtual keyboard.  This is very practical and efficient for manual testing, by eliminating mis-taps, for example, as well as allowing the tester to take screenshots or record video to share defects with developers.  This is a requirement for automation on real devices because all automation tools reside there.

 

8.Choose the level of test integrity you are comfortable with and stand behind it.  There are three primary methods to enable remote control of mobile devices: use of third party tools, use of hooks compiled into your application, or a combination of both.  There are pros and cons to each method and it can be difficult to determine which option is the best.  Selection of a method is a highly controversial subject, with testing leaders debating the level of integrity that is lost with either method.  You must choose the method or methods that best meet your organization’s needs, and then minimize any potential risks.  For more in-depth information about this topic, please read “Remote Control for Mobile Devices“.

 

9.Use test automation.  Evaluate available mobile test automation tools using criteria tailored to your organization’s needs.  Don’t get caught up in feature lists or record/playback demos!  A vendor with the most checkmarks on a feature list may not be the best fit for your organization’s business requirements.  For example, consider what it takes to update and maintain an automation script after it is written as part of the total cost of ownership. For more detail on test automation, download, “Mobile Application Test Automation“.

 

10.Adapt your approach to your methodology.  Adapt your testing approach to your development methodology. Are you Waterfall or Agile? Many of the decisions you make about using remote control tools, test lab setup, and even manual vs. automation testing, should be made with your company’s preferred development methodology in mind. Why? Because how your mobile app is developed impacts the feasibility of the testing approach that works best for your company.