Quantitative vs Qualitative Analysis
A common trait between humans is we have emotions. Decisions we make – we like to think it’s based on reasoning and logic but, in fact, it’s derived from intangibles like feelings and emotions. That’s not not a bad trait and certainly shouldn’t be considered as an appendix from our ancestors; on the contrary more advanced creatures are more emotional than primitives, human being the most emotional of all helping us to make decisions quickly.
However, when building products for our users we often forget this behavioral aspect.
We get so absorbed in data analysis and building next set of features that we forget about the poor user’s feelings and emotions. Don’t get me wrong, quantitative data analysis is an important aspect to understand key metrics such as installs, signups and retention, but to get the full picture, understanding user’s behaviour through qualitative user analysis is even more important.
A combination of both quantitative and qualitative tools will help you understand your user’s behavior in depth. Let’s look briefly into both of the analysis and how they can interplay to help you understand your users better.
Quantitative vs Qualitative analysis
We will differentiate qualitative and quantitative analysis through a metaphor:
Let’s say a close friend of yours took a vacation to Nepal and when he came back, he told you his story of the mighty Mt. Everest. This is like traditional quantitative data analysis.
In contrast, now when he takes out his iPhone and shows you pictures and videos from the trip, you have a better understanding of his experience and that’s qualitative visual analysis.
Quantitative analytics deal with data that can be measured. While iterating on an app, this data is used to measure user actions and generate reports such as funnel analysis, cohort analysis to understand engagement, conversion and retention. Google analytics, Mixpanel and Kissmetrics are the most popular ones. Although all of these analytics give the same services in the surface level, but at the core they have their own individual strength.
Quantitative analytics are great at tracking metrics that can be numerically depicted answering “how many” and “how much”, but they don’t reveal the full story. It lacks in-depth insights about your user’s behavior and this is where qualitative analysis comes in play.
Since there is a great deal of information about quantitative analysis, we will focus on qualitative analysis to showcase the features that can be used to answer ‘Why’.
“To design the best UX, pay attention to what users do, not what they say. Self-reported claims are unreliable, as are user speculations about future behavior. Users do not know what they want.” - Jakob Nielsen, First Rule of Usability? Don't Listen to Users
Qualitative analysis provide insights that are not found from the use of Quantitative analysis. For example, Quantitative analysis tools can provide insights into your app signups, app visits and bounce rate but you will not get to the depth of why your users are leaving your app as soon as they open it or why your new sign up is not converting into sales. To answer the reason behind “WHY”, you need a deeper understanding into the user’s behavior in your app and this is where qualitative UX analysis comes into the frame.
Qualitative analysis is about observing a real user to understand what people do inside your product and get insights on their actual behavior. It empowers you to see how a user uses your product and understand where they are struggling.
The popular tools for qualitative analysis include CrazyEgg and ClickTale for web and UXCam for mobile.
- UXCam: UXCam allows you to see how users are using your app and understand where they are struggling with Session replay, heatmaps and UX analysis
- CrazyEgg: CrazyEgg a visual analytics service for web provides you with click and scroll heatmaps.
- ClickTale: Going a step further to CrazyEgg, ClickTale provides you videos of your website heatmaps. With their session replays you can look at how customers use your websites.
Key features of these UX analytics tools:
Heat Maps allow you to understand how users are interacting on each screen by aggregating all user gestures (taps, swipe, scrolls).
This allows you to see which part of your screen is being used the most and which needs reworked, enabling you to redesign and optimize the UI for conversion.
A picture is worth 1000 words or in this case a video is worth gigabytes of data. Qualitative analysis tool provides user recordings that enable you to see how a user is using your product along with all touch interactions through session replay.
This enables you to understand where in the product your users are struggling.
Quantitative tools answer your ‘What’ and Qualitative tools tell you ‘Why’. Both quantitative and qualitative analysis have their own strengths. While quantitative tools allow you to understand the key metrics, qualitative analysis allows you to understand user behavior.
Think of it this way, quantitative tool are great to understand where to look for issues, qualitative tools show you, ‘it’s here’, use both in parallel to optimize your app.