How to use data to challenge the HiPPO

Identifying the HiPPO

In a mobile app company, a HiPPO (Highest Paid Person’s Opinion) could come from an executive, president, or board member. The HiPPO is anyone who has an opinion that is highly valued by a larger group. Their opinion is bolstered by expertise but more often by the position or title they hold.

The HiPPO effect and authority bias in tech

The HiPPO effect is a term that describes a decision-making process where the HiPPO’s opinion(s) heavily sways or determines the outcome. The HiPPO will impose a solution to the group often based on their ‘gut’, which is based on previously validated decisions in different circumstances. Meaning that the idea may be supported anecdotally, but there is no specific outcome nor a way to measure that outcome.

You might also be interested in: How to make and justify design decisions with data

HiPPO titles alone — often CEO, CPO, VP Product, the investor — will amplify their opinion’s impact on their fellow stakeholders due to authority bias. This is our natural tendency to exaggerate the opinion of authority figures. According to a Lindsay Morgia article on Ness Labs on the subject; authority bias provides streamlined decision-making. However, the HiPPO may not have all of the information or knowledge required to make the right decision.

“...authority bias is a shortcut our brains use to save time and energy making decisions… However, problems arise when we rely too heavily on this heuristic and assume certain authority figures have more knowledge or skills than they actually do.”

In short, the HiPPO can voice their opinion and everyone in the room may simply assume they are correct. Due to respect or even fear of speaking up, other team members are reluctant to voice objections. 

It would seem that we’d moved past the HiPPO effect in the tech industry. There are endless unknown unknowns in the mobile app industry where we’re just on the brink of a seismic shift in user behavior. What worked in the past may not work in the future. 

As businesses move to mobile-only or mobile-first, product teams are working in uncharted territory. The value of analytics, user testing, and usability testing on the unique data points that mobile provides has been proven but demands more attention. For some businesses, a few dozen user testers are not enough to justify a change that could take 4 months to execute.

Many business leaders think they can’t afford to wait for research due to budget constraints or market demands that are moving at a breakneck speed. But in a work culture where we’re supposed to move fast, fail fast, authority bias can lead teams to do the latter. 

A classic HiPPO in product management tale

A few years ago, I was tasked with improving the activation of users for a mortgage servicing app with a small credit union. The current application onboardingcam.com/blog/10-apps-with-great-user-onboarding/ requires linking to your end user’s bank account for wire transfer of funds. This requires account verification.

Since we were turning existing customers of the bank into mobile customers, in theory, the onboarding and activation part of building the mobile app should have been easy. We already had a customer base. 

The original verification process: The user could verify their account using only their mobile banking login credentials and online pin number. It took seconds to verify accounts.

The challenge: However, six months after the release of the app, the product team saw that a majority of the banking customers were not moving past the sign-up phase of onboarding. For a majority of users, their mobile banking accounts showed little to no activity. 

Understanding the problem: The product team conducted interviews with existing customers about their experience with the mobile app. The recordings from our sample of 50 users found an overwhelming amount of evidence that the majority of users (who were aged 40-65), weren’t apt to trusting services like instant account verification using only login credentials.

Since the business was a small credit union, the end user could have a generational bias against sharing private bank account information on a mobile phone. 

Example of micro-deposits in banking apps.

The product team proposal:  We proposed that we should change the account verification process to micro-deposits. Qualitative data showed that the customers were in an older demographic, aged 40-65, and for many, it was their first mobile banking app. It would take a minute or two rather than a few seconds to verify the account, but it felt much more secure. 

The HiPPO effect: The founder and CEO felt like the sample size of the user interviews was too small to justify changing the most important part of onboarding. He also said that faster verification was commonly used on competitor apps (e.g. Guaranteed Rate, Rocket Mortgage, SoFi). He advised to find alternate solutions to the lack of active mobile app users. Despite the user interviews, he felt like we didn’t have enough conclusive evidence for this change. 

The challenge: Based on our gut, we knew that these more modern fintech companies using instant verification for banking login were used by younger demographics that were accustomed to mobile banking. 

We need to show that the current account verification process is the cause of customer drop-off for the older demographic that isn’t used to instant verification. Our challenge was to present both the what and the why. We want to get to a point where we can test whether micro-deposits can increase activation rates. 

You might also be interested in: The pocket guide to app analytics

Starting with what matters: KPIs and hypothesis

I needed to speak to our head of product and CEO to propose this huge change in onboarding. To start, I looked back on the KPIs that were set for the start of the quarter. I knew to always loop my argument back to these KPIs. 

1. KPI 1: Increase new user onboarding - the acquisition of newly registered users

2. KPI 2: Decrease registration abandonment - Abandonment represents users who give up on the registration process before completion.

While simply using banking information and a pin will speed up user onboarding, it will increase registration abandonment due to a lack of trust. Instead, we should use micro-deposits to verify since it is a much more secure way of account verification. Micro-deposits are very small transactions that are deposited into the end user’s account and then identified by the account holder. They are a fairly common method of account verification in this industry.

Our hypothesis:

We believe that changing the account verification to a more traditional method of micro-deposits will increase the onboarding completion rate for our target user demographic (existing credit union customers aged 40-65) because it is perceived to be a safer method of account verification compared to logging in with only account credentials. 

In this product hypothesis example, I used the formula: 

• We believe [new feature solution]

• Will [direction of change]

• For [target user]

• Because [your why]

Back your hypothesis with data

Armed with data, prepare for your meeting by showing your results to people who represent more than the tech team, ask customer success and business analysts. 

As mentioned, the CEO thought that the sample of users was too small to make a sound decision. So we used an analytics tool to show 10,000 user sessions. At this point, there’d be no denying that there was a pattern that supported our user interview research. 

First, we identified where the user drops off in registration with session funnels. This is a recreation of our graph but you can see here clearly that there is an 86% drop at registration. 

Drop off

With a session funnel we tracked the onboarding and registration processes that you expect the user to complete in a session. In the graph,  you will get the count and percentage of all the sessions where a certain action is or is not completed. 

Funnel example on UXCam.

Quit rate

We clicked between the steps to better understand what happened during the sessions. This way we could drill down on the specific list of sessions where the verification process was abandoned. We were able to see overall data on the top of the screen e.g. rage taps, crashes and so on. On this screen, everything looked alright. So far, it looked like there weren’t any technical problems causing the drop e.g. bugs or freezes. We went into the heatmaps dashboard and saw there was an unusually high quit rate.

Quit rate example.

From here, we saw that the login page that uses just mobile banking login data had a high quit rate of 75%. The engagement time was also lower than average. A quit rate of 50% or higher (based on visit counts) should raise some flags. 

To dig deeper into this problem, we analyzed the heatmaps of the login page and we saw that users were clicking around the login field but not logging in. 

Heatmap example.

They were swiping through the splash screens but once they got to the login page, they clicked around. 

So now we have quantitative data to prove our hypothesis that we’re losing users at the login page. To complement it with qualitative data, I clicked into the session replays and watched the users tapping around the verification page that requested their banking information and pin number. From there, they tried to go back, possibly looking for alternate verification methods. Ultimately they just closed the app. 

You might also be interested in: Complete guide to mobile heatmaps

To get the bigger picture, we looked at the screen flow. We saw they were going from login to quitting. It was becoming clearer that users were stopping at the login page. We also saw that some users were logging in just fine, but most were not - so this ruled out many technical issues.

Funnel example.

I collected all of this data into a dashboard and shared it with my team members. We were ready to present this data to prove that the bottleneck is in the account verification. 

This was the route that I used. But there are several other ways to identify the behavioral characteristics of onboarding, these include (but are not limited to):

• Quit rate

• Drop off rates

• Exit screens

• Onboarding completion rate

• Pages that the user drops off during onboarding

• Custom events such as a password failure

• Rage clicks

• App store reviews

• User feedback surveys

Visualized in a clear and easy-to-digest manner, these metrics can be powerful tools to tell the story of what the user really wants versus what we think they want. 

Preparing the meeting with the HiPPO

• Timing is everything. There’s lots of literature out there on when the best time to set up a meeting is, but simply ask the HiPPO what time of day works for him and wedge your meeting in there. 

• Invite allies and people the executive trusts. When setting the calendar invite, who needs to be in this meeting? Who are your allies? Do you need support from tech leads or business intelligence to support?

• Prep the big picture. Has the HiPPO been made aware of the current problem? Be prepared to answer how long it took you to do this research, how long it will take to implement your idea, and what resources you’ll need. 

Presenting your research

• Prepare easy-to-consume quantitative and qualitative data that can tell the full story of your decision. Include: 

  • User research: Sentiment from customer interviews and reviews.

  • Qualitative and quantitative data: Here is where you can shine with the data we discussed in the previous section. 

  • Possible threats: Predict the consequences of implementing an alternative solution to the one you have planned.

1. The HiPPO agrees - yay! Assure them that this is going to be the best for the customer. 

2. The HiPPO persists - rinse and repeat. Give advice and if your suggestion is not taken, have everything documented, at the end of the day, you did your job. Although frustrating, this may not necessarily be the hill you have to die on. Oftentimes the metrics will speak for themselves. Keep a tight eye on your live dashboards, and if the results do not align with the product goals, bring that to the attention of the larger group. 

Sadly, that was the case in the example above, ultimately we had to concede to the HiPPO and we implemented their solution in haste. Unfortunately as a result there was an increase in abandonment directly contrasting our product goals. Fortunately, this was an excellent retro experience for all involved. 

If the results also don’t change the HiPPO’s ways, then this could be an embedded culture mismatch. Consider corporate culture, are you working for a real product org? If not, can you help get them there? 

Etiquette

• Be respectful. First things first, this business would not exist without this person.

• Honor the timeline that was provided. Be efficient with a time-bound agenda.

• Stay open. Make it clear that product suggestions should always be welcomed, but also make it clear you cannot commit to anything without doing the necessary research. Ensure that you will set up a follow-up meeting, and also be sure to provide a timeline.

• Stand for the customer. Dive into how this change could impact the customer (what they want to do vs what’s actually happening). 

How did the story end? 

When the product team presented the research, we proposed the new idea: Micro-deposits to verify accounts. It was faster than a manual call for account verification and much more secure than logging in with your banking credentials. Micro-deposits would reduce that processing time from days to minutes. We’re currently working on the solution, and when we have an MVP, we’ll use the same methodology as above to see if logging in has improved it and reiterate. 

Ask questions 

As you can see, the hypothesis building and research part is the heaviest section of preparing for your meeting with the HiPPO. Armed with this, you’re going to show that you’re professional, real quick.

The goal isn’t to protect your idea, but to discover what will actually deliver the most value. You’re asking questions to the HiPPO to verify whether their idea can actually make a positive impact. 

Plan to ask the HiPPO questions that you could answer yourself. If they know the answers, perhaps they have put some time and research into it themselves, and it's a validated request. 

You might also be interested in: Why you should be maximizing for CLTV rather than CAC in today’s market