In this post, we will discuss the who, what, where, and why of user testing.
User testing is a term that floats around the universe of product managers quite often. However, it seems there is a lot of confusion out there as to what exactly user testing is. We often hear it used broadly to describe any sort of testing that occurs, whether it involves focus groups, market research, or other forms of testing. So we figured we’d start with the basics and work from there.
What is user testing?
User testing is the process by which we examine the ways our target users interact with our product. During the user testing process we ask our target customers to complete tasks, typically while being observed by a researcher so that we can see the barriers they encounter or if they experience confusion along the way.
What undergoes user testing?
Any product that will be released into the market should undergo multiple stages of user testing. It is an integral part of the development process beginning with the birth of the idea. If you cannot get people excited about the feature or product you are releasing with only the idea, it is going to be difficult for your marketing team to bring users. The user testing process should continue throughout development so that you can optimize as much as possible before bringing the product to market.
Who performs user testing?
While the actual tests are typically performed by a researcher, the data gathered is used by the product teams. As user testing is evolves, the importance of the researcher’s role has been significantly reduced. Product managers are now able to run the tests and receive the data firsthand from the users. This allows them to acquire data without any external input, thus controlling for bias in their decision making process.
On the flip side, if a product manager is overly attached to a project, they may not be able to accept criticism without a bias. This can be detrimental to the data analysis and may result in the dismissal of important customer experience feedback.
Additionally, product managers who are not involved in the test may feel the need to prompt the user to certain actions, compromising the results. Thus, our recommendation is to cut the middle man and assign testing to the product managers for optimal results. This removes bias and the option for the product manager to prompt the user.
When is user testing performed?
This is an extremely tricky question. Often times, user testing is performed prior to a product’s release. It’s almost comparable to the crash tests performed prior to an auto release. You are ensuring that the product you are going to be releasing is as stable, safe, and easy to use as possible.
However, user testing should not end once your product is released. In fact, we could argue that once the product is released, user testing becomes even more important. Once a product is live and in the market, any flaws become dramatically more noticeable. Any errors in your product are not just a simple fix anymore, they are now live with your customers and interrupting their experience. Tools like Jaco can help you identify these flaws and fix them before they become too big an issue.
How do you perform user testing?
The type of user testing you are going to perform depends on the product stage. For example. If the product is very early stage, user testing can be quite simple and accomplished in a quick Q&A session. However, as the product begins to take shape, the test should become more sophisticated in order to observe how users react to a more complex model.
User testing example: Airbnb
Let’s use AirBnB to illustrate. During early phases, user testing would be a simple test of the idea: “How would you feel about renting someone’s apartment or house instead of going to a hotel?” As the product grows and adds more features, you might give your test user the following task: “Book a one bedroom apartment in Manhattan for July 8 – 15 at a total cost of $1,000. The apartment should have wifi, air conditioning, a hot tub, and served breakfast.”
Observe, and most importantly, do not interfere, as the user attempts to perform the task. They will have to use the entire platform as they attempt to make this booking. The gold nuggets are are the different emotional states you will be able to observe on your user. Those might range from frustration when they cannot find a specific feature, to elation when they are able to complete the task successfully. You will then be able to pinpoint the problem areas and make the appropriate optimizations to ensure that frustration does not occur in the future.
This is user testing in a nutshell. It is a vital part of the product development process and should be done carefully and in the appropriate environment. Try to user test with as many users as possible, both for statistical significance and to hear a variety of views and opinions. Remember, even when the product is released — optimization is an ongoing process that we as product managers should always keep an eye on.