This is an excerpt of a new book I’ve written with Wes Bush called Product-Led Onboarding™: How to Turn New Users into Lifelong Customers. You can buy it now!
If you measure the wrong things, you’ll improve the wrong thing with your user onboarding.
In this article, I'll share three important KPIs (key performance indicators) to measure success milestones that you need to be tracking if you're looking to improve your product's onboarding.
The Danger of Tracking The Wrong Things
First, Why is it important to track the right things?
Let me tell you a quick story.
Google Maps can be quite a marvel.
The vast majority of the time, it gets us where we need to go without having to think about it. We just plug in our destination, and boom, we’re there. So many of us have become so reliant on it that when it fails, everything goes awry.
I experienced this firsthand with my wife Joanna on a road trip from Toronto to Montreal. Unfamiliar with the streets of Montreal, I relied heavily on Google Maps to find our hotel. Lo and behold, my phone lost signal after exiting an underground tunnel, and I received that dreaded error message, “GPS lost signal.”
We ended up driving over a bridge that led to the US-Canada border. That wouldn’t be so bad if we both had our Canadian passports, but we didn’t!
Immediately, we both started freaking out. How are we supposed to explain to the Canadian Border Agents that we’re not illegal immigrants trying to sneak into the United States of America, a.k.a. the land of the free?
Of course, we completely overreacted. The Canadian Border Services Agency officer we spoke with was understanding. After showing him our Ontario driver’s license and explaining the Google Maps failure, he laughed it off. All we had to do was exit on a ramp that led us back to where we needed to go.
This is a harmless story, where not knowing your destination can lead you down the wrong path. But it doesn’t always end well, especially with the cross-functional, highly-visible effort of improving user onboarding.
In my last article, I provide a step-by-step process to help you get a clear picture of what your users’ desired outcomes are along with the functional, emotional, and social jobs they are hiring your product for.
The next step is to define what success looks like for your user onboarding experience. There are three moments that matter the most to measure onboarding success:
- When users complete the signup process
- When users experience the value of a product for the first time
- When they begin to use it consistently
These three success criteria are tied to key milestones in the user onboarding: the Moment of Value Perception, Moment of Value Experience, and Moment of Value Adoption.
Success Criteria #1: The Signup
The first onboarding success criteria is getting users to complete the signup process. It may be a signal of a user’s interest. The higher the friction is during the signup process, the higher the signal of interest.
For example, if you require a credit card upfront before someone can start a free trial (a.k.a. an opt-out free trial), you’ll likely get fewer but higher-quality signups who are more likely to convert. Based on my experience, the trial-to-paid conversion rate of opt-out trials is as high as 60%.
On the other hand, for opt-in trials that don’t require credit card upfront, you’ll likely increase your signup rate, but your trial-to-paid conversion rate could be as low as 5%. If your intent is to get more people to try your product, the best option is to make your sign-up process as frictionless as possible. This means not requiring credit card details upfront with no strings attached. You’ll get more people to try your product, but you have to qualify them better, and try to win more of them.
Regardless of whether you have an opt-in or opt-out signup experience, a stronger signal of onboarding success is when users first experience the product’s value.
Success Criteria #2: The First Strike
The second onboarding success criteria is helping users achieve their desired outcome, or Customer Job, as quickly as possible.
In Wes Bush’s Bowling Alley framework, this event is called the “First Strike.” In 10-pin bowling, this happens when all ten pins are knocked down with a bowling ball. (We’ll go deeper into the Bowling Alley framework in the next chapter.)
In user onboarding, the First Strike is a necessary product action users must take to accomplish their desired outcome or Customer Job. Let me give some examples.
- For Canva, it’s exporting or sharing a finalized design
- For Zoom, it’s hosting or attending a Zoom meeting
- For eCommerce stores, it’s the first purchase of a product
For some companies, this First Strike isn’t so obvious. Think about Facebook. What’s their First Strike? Is it a user liking, sharing, or commenting on posts?
Since those actions require users to have a few Facebook friends, the most important product action is to add a friend. That’s why Facebook prompts new users to connect with friends as quickly as possible. This happens early on in their onboarding process.
For “all-in-one” products that solve many problems, recognizing the First Strike is even more tricky. Imagine identifying the First Strike for a Swiss Army knife. Is it the knife, pen, screwdriver, compass, scissors, saw blade, fire starter, or laser pointer?
Many all-in-one B2B SaaS products can essentially be considered digital Swiss Army knives. How do you determine the First Strike in these cases?
This is where context and segment matter. Depending on the context of use and the segment, there could be different First Strikes. For example, users can accomplish a lot with Intercom:
- Guide customers through their first steps, as well as highlight what's new with product tours
- Build mobile carousels to onboard new app users
- Create smart bots to qualify visitors on a website
- Connect with website visitors and answer their questions in real-time
- Manage customer support requests and questions
- Make it easy for customers to serve themselves by sharing relevant help articles
So, what is Intercom’s First Strike?
It’s hard to decipher at first glance. But by segmenting Intercom’s users, you learn that users hire Intercom for three core Customer Jobs, all aligning with their three product lines:
- Conversational Support product: provides human, self-serve, and proactive support to increase customer satisfaction. A possible First Strike for this Customer Job is the first time users respond to and resolve a customer request.
- Conversational Engagement product: increases the engagement and product adoption with targeted in-app and outbound messages. A possible First Strike for this Customer Job is the first time users respond to an in-app or outbound message.
- Conversational Marketing product: acquires new customers quicker by responding to questions from prospects in real-time. A possible First Strike for this Customer Job is the first time users respond to a question from a website visitor.
As you can see, the First Strike is closely tied to your product’s Customer Job (or Jobs, for more complex products). It’s the first measure of success that new users are on the right track.
Success Criteria #3: The Product Adoption Indicator
The second measure of success is the moment users start using a product consistently.
One of the end goals of onboarding is helping users embrace new habits with a product. Habit-forming user onboarding requires users to experience the value of a product more than once. If new users have used the product enough times, they’re more likely to continue with it going forward.
As mentioned in Chapter 1, for Slack, a team is not successfully onboarded until they’ve sent not one, not 10, but 2,000 messages. It’s at this threshold where they’ve found the teams who are likely to continue using it.
Since this onboarding success metric is a leading indicator of user retention and product adoption, I call this the Product Adoption Indicator, or PAI for short. It’s an early but strong signal that users are likely to continue using a product going forward. They have embraced the product and are very unlikely to return to their old habits.
This concept is not new—others call this the “magic number.” Here are some well-known examples:
- At Facebook, users who add seven friends in 10 days are more likely to continue using Facebook
- In the early days of Twitter, the rule “Users who follow 30 people” was a retention and growth driver
- At Dropbox, users who added a file in one Dropbox folder on one device were more likely to add more files to their Dropbox
Notice that the PAIs for each of these products are closely tied to retention.
Once users reach this point, you’ve set them up for success. They’re now ready to take the next step in the product’s customer journey. They’ve completed the initial loop of the user onboarding process: they’ve perceived, experienced, and adopted your product for the first time.
Good PAIs have a few common characteristics:
- PAIs should be leading indicators of user retention: The PAI is the onboarding team’s “canary in the coal mine,” which means it provides early warning signs for miners of dangerous gases in a coal mine. With PAIs, you can predict with some confidence early on of the likelihood of users sticking around to continue using a product. In other words, they’re a strong signal that users have started forming a habit of using the product.
- PAIs should focus on the repetition of one key product engagement action: One of the goals of user onboarding is to help new users build habits using a product. Since repetition is key for building habits, users need to use the product a few times before it finally clicks for them. The PAI is tied to the product’s desired outcomes or Customer Jobs (i.e., when a team has sent 2,000 Slack messages).
- PAIs should be easy to understand and communicate with others: PAIs should be simple enough to remember and communicate with the entire company. Facebook employees “talked about nothing else” but seven friends in 10 days; it was their single, sole focus. Instead of layering several user actions into the PAI, such as the number of likes, comments, and status updates, the company emphasized simplicity so it’s easy to remember and share within the organization.
- PAIs should be time-bound: Ideally, you want users to complete the PAI within a specific timeframe. With Facebook, it’s adding seven friends in 10 days. Slack’s and Twitter’s PAIs are not time-bound. But, having a timeframe can help your team identify if new users are off-track so you can adjust the user onboarding accordingly.
- PAIs should come early in the user’s journey. You want the ability to identify whether new users are on the path to success as early as possible. If the PAI occurs weeks later after signing up, you’ll have fewer data points since many of those users will have already churned.
How to Determine Your PAI
If you want to figure out your PAI, I’ve provided our 5-step process to do that. Just go to productled.com/pai to get that guide.
This is an excerpt of a new book I’ve written with Wes Bush called Product-Led Onboarding™: How to Turn New Users into Lifelong Customers. You can buy it now!
