Collecting user feedback when conducting user testing is key in improving your product. The problem is that your users are most likely lying to you about how they use or feel about your product.
They’re not doing it because they’re evil. They are doing it because they can’t help it. There are 4 common reasons users fail to give you real answers:
- User feedback is skewed because they want to make you happy.
- They don’t pick up on their own dishonesty – it’s hard to self-document your behaviour.
- They rationalize their behaviour in retrospective. Which means they lie about intent and how they feel about the product.
- They want to feel better about themselves – when they experience difficulties using your product they don’t start by blaming you. They start by thinking “am I not smart enough?”. Then, when they report back, they will try to make themselves feel smarter, and might fail you.
Remember, Self-awareness is hard. only the Dalai Lama has enough insight on himself to understand his own behaviors. “Users can try and tell you what they did, but it’s hard for them to identify why.” Click To Tweet According to this
According to this Neilsen Norman Group, “while it’s not realistic to use the full set of methods on a given project, nearly all projects would benefit from multiple research methods and from combining insights.”
There are different elements to take into consideration when planning and choosing your user testing method.
One of the first big decisions you have to make when planning your user testing is choosing between running a moderated VS. unmoderated test.
This means that you have to choose between being present and observe your users as they use your product or not.
Get the Most Out of Your User Session Recordings: Download our FREE eBook about it
While it’s tempting to try and be as present as possible to see them react in real time – you are risking in skewing the test results by due to “The observer effect”
In science, the term observer effect refers to changes that the act of observation will make on a phenomenon being observed. This is often the result of instruments that, by necessity, alter the state of what they measure in some manner.
We have compiled a list of ways to uncover the what and the why of how your user use your product while minimizing “the observer effect” and getting the most truthful accurate answers.
Using these methods, your users can only tell you the truth – because you’re going to uncover their actual behaviours.
1. Recording user sessions
Imagine an undercover situation where you,the product manager or designer, are able to watch over the shoulder of an unassuming customer as they use your product.
You can see exactly the pattern in which they click, how long they spend in each space, and where they stumble or get confused.
Now imagine that this situation happens with all of your users, and your eyes have special recording devices to allow you to re-analyze the video later to compare to other users experiences. That is some RICH data.
By using a user testing tool that acts as screen recording software for your user’s screen, this situation is possible without creeping over a shoulder.
You can see directly through the eye of the user in the actual time that they are using it. You can see what their instinctual reflex is, and the process in which they maneuver your site.
This method alone can dip inside the subconscious of the user and tell you so much about browsing behavior in the user’s natural habitat.
Just remember that you need a tool that won’t hurt your user’s experience (don’t ask them to install anything in the process).
Feel free to check out our feature page to see how we user session recordings.
Now, it’s just a matter of paying attention and analyzing the recorded sessions.
With Heatmaps, you can be the eyes of your user.
You can see the areas that are most attractive to the user, and what sections aren’t even touched.
Heatmaps are a visual indication of areas the mouse hovers or clicks most (in red) and areas that are not (in blue).
Here is an example of a landing page heatmap taken from a client at Techwyse:
In this case study, the “red” parts in the “before” picture are not even linked buttons. That means that users were clicking on nothing and the landing page did not have a clear action.
The solution was to make a clearer CTA (call-to-action) by making it front and center and easily recognizable.
Notice that the CTA in the “after” picture was to “Call now” which you can’t see behind all that red, hot attraction. In other words, the change worked and probably would not have been seen without the use of the heatmap. Heatmaps can give you a better idea of where to put your advertisements or CTA. And, paired with recordings of user sessions you can be the hands, the eyes and the subconscious of the user.
If you would’ve asked your user – how easy was it for your to find the “Call now” button, they wouldn’t tell you how many times they’ve clicked the un-clickable areas of your site before actually hitting the button.
3. Post usage surveys
Post usage surveys may be old, but they are deemed useful time and time again. This is where we enter the conscious minds of our users and give them a voice.
We are able to compare our observation of their experience and put words to their actions.
With surveys, we are able to assess a user’s preferences, attitudes, characteristics, and opinions. You can ask the user directly what they were looking for, and if they found it with user intent studies.
Pro tip: Users may not give you the truth about their experience. But, you can match answers from the survey with the actions you witnessed and uncover their attitudes and opinions of the user experience (which is just as important).
This article about “the problem of the self-report in survey research” hits the nail on the head with these examples. In the same way that survey questions about voting can be compared with voting records,or questions about library card ownership can be compared with library records, we can detect honesty and dishonesty by comparing our data with the survey responses.
Why surveys still matter? Because they can uncover what was going inside your user’s mind when taking action.
If you want to create better insights from your surveys – base your questions on the behaviours you’ve already witnessed on your session recording and heat maps.
4. Look at their behavioural analytics
You have now visualized the experience through the user’s eyes, hands, and minds and have seen where the problems lie.
Do the observed user experiences correlate with the data in your analytics report? Are most of your users experiencing the same pain points that you witnessed in your recorded user sessions and heat maps?
While quantitative data can’t answer anything on its own, numbers are imperative to generating proof of a concept – especially when convincing stakeholders of a product re-design.
5. Give them specific tasks, and see if they manage to do them
After utilizing the above four methods to uncover navigation problems, you can now target these issues by getting users to navigate a specific feature.
Let’s say that the above studies pointed to confusion on the payment page. Now you can identify the specific barriers users are encountering and fix them.
But, this method is not at all a solid answer to all your problems. The only way to dive deeper is to know the broader data first.
Giving specific tasks to users is a great tool once you already know the pain points that need fixing. The more you can observe without any prompting, the better chance of understanding the true user experience.
By providing them with specific tasks you can track improvement in specific frictions you’ve witnessed in your product and measure improvement.
These 5 methods of user feedback can uncover the most important information about the user’s experience.
Uncovering a user’s behavior and attitudes, while also collecting both qualitative and quantitative data gives you all angles to optimize your product usability.
You want the opportunity to observe a user in their natural use of the product, and then attack the problems found by digging deeper and directly asking the user about their experience.
Combining insights can make you the eyes, hands, and minds of your users. Now is your chance to take a walk in their shoes and make the right changes for the best results.
How are you running your usability testing? Let us know in the comments.