How can you combine product strategy with analytics to build a better product?
In this video and blog post, I'll share how to use data to help you set your strategy. I said "help" because we're not yet to a point where the data can tell us exactly what to do. The world of B2B SaaS is very complex. What we're doing today is at the intersection of art and science, combining them to build compelling product experiences.
Why it is so important to consider the art and science of product development
Let's start with a quick history lesson to demonstrate why it's so important that we think about both the art and science of product development, especially in B2B SaaS.
Any idea what this growth curve represents?
It's the growth of the internet, and it shows that took eight years to get to 100 million users. It was one of the fastest adoptions of technology at the time.
And what do you think this growth curve might be?
This chart shows how long it took Fortnight Battle Royale to get to 100 million users: just 3 months on iOS alone. In that period of time it grossed almost $1B, which speaks to how quickly amazing product experiences can transform industries.
We're seeing this happen in B2B, in media, and ecommerce. Slack, Netflix, and Amazon are changing the game in their industries, and they're doing it by building transformative and innovative product experiences that their customers love.
How are they doing that? And what can you do to bring that back to your SaaS company so you can also build these types of experiences?
I can tell you what they're not doing. They're not going through business school marketing exercises. They're not gathering their board for an annual strategy meeting. And they're not hiring marketing strategy consultants to tell them what to do.
Instead, all of these companies are winning through their products: they are product-led. They understand that their competitive advantage in the industry is the experience that they're building and delivering through their product.
Successful product-led companies share three key things:
- Set a vision and create clarity through a clear and measurable North Star
- Set a strategy, integrating behavioral science into their decision-making frameworks
- Results-oriented, accountable to outcomes
Let's look at how B2B SaaS companies are doing all three of these things.
1. Create a vision
A great product-led B2B SaaS company sets a vision to create clarity around what matters most.
There are a lot of definitions of what "vision" is and how to approach setting a vision for your company. I like to define vision as the measurable world we want to create.
Let's start on the second half - what does the world we want to create look like?
Fire Mario may not seem to be relevant to a product manager trying to create a vision. But he really is a perfect analogy for marketing strategy. The User Onboarding Group created this graphic to explain:
Small Mario represents the customer, and where they are today. The fire flower represents our product. Fire Mario represents how awesome Mario can become if he has access to our product.
When most people think about product vision, they think about the middle part, the fire flower. But actually, product vision is about small Mario and the pain he's in, and Fire Mario with his newfound abilities gained from using our product.
It's really important as a product manager to focus on both customer pain and the outcomes we want to drive.
A great example of product vision is Spotify. Spotify's vision is simple but powerful - give people access to all the music they want all the time in a completely legal and accessible way.
If we break this down, we start to see a lot of small Mario and Fire Mario.
|Fire Mario||Small Mario|
|ALL the music||Purchased music|
|ALL the time||When the radio plays it|
|Legal access||Illegal torrents and downloads|
|Accessible||Need a compatible device|
Nowhere in this vision statement do they mention features. It is all about the ultimate outcomes we want the customers to have.
You might be wondering what product vision has got to do with analytics? The reality is, without a compelling vision, strategy, data, and everything else doesn't matter. To scale a company, you need to know where you're going.
What is a North Star?
You're probably familiar with the concept of a North Star. It is a leading indicator of future success. But, how can you put North Star into practice to help set a strategy for your SaaS product?
A North Star is like the guiding light that moves your company from the vision to the impact you want to create. Having worked with hundreds of product-led B2B SaaS, we've learned that there are the three steps you must take to set a compelling North Star:
- Define the game you're playing
- Identify a North Star that drive business outcomes
- Choose input metrics that are leading indicators of that North Star
As a product manager, you must follow each of these three steps to set a North Star that drives your strategy.
To begin, identify the game you are playing. If you think you are playing checkers, but all of your competitors think you're playing chess, you're going to have a really hard time winning.
We've found through our research that there are three different games in SaaS: the games of attention, transaction, and productivity.
The game of attention
The game of attention focuses on delivering value through the amount of time the user spends on your product. Typical SaaS product North Stars for the game of attention include the total time spent engaging with content, streaming hours, etc.
Companies such as Facebook and Netflix are in the game of attention because they want users to spend as much time as possible on their sites.
The game of transaction
If your product's purpose is to have users engage in a transaction, you are playing the game of transaction. A few typical North Star metrics for transaction-based SaaS include the number of purchases completed, the number of seat upgrades purchased, or the number of searches completed.
Some examples of companies playing the transaction game include Amazon, United, and Walmart.
The game of productivity
The productivity game involves getting users to some sort of task success. Typical product North Stars for the game of productivity include the number of records created, number of messages sent, or the number of queries completed.
The classic example of a B2B SaaS company that plays the game of productivity is Slack, and others like Amplitude, Asana, and Salesforce.
Once we define the games, it's about coming up with the best North Star for the game you are playing.
How to set a North Star
To create an effective North Star for your company, you must design it around your company's goals within your game. Even within the same game, not all North Stars are the same.
Your North Star must align with these three things:
Measure customer value
We don't want to see things like opening the app or clicking an ad. Those might be important for your SaaS marketing strategy, but they do not add value for the customer.
Aligned with product vision
What is your Fire Mario? Make sure this is a great measure of your success taking users from small Mario to Fire Mario.
Leading indicator of revenue
The North Star metric should not be revenue because it's actually a lagging indicator. Instead, use a product outcome that leads to revenue so you can get your marketing and product teams aligned behind it.
A word of warning when setting your North Star - do NOT make your North Star a vanity metric! Typically, vanity metrics are "top of the funnel" metrics. It may be important to measure things like your number of active users, but a good North Star metric is deeper into the funnel.
Postmate's North Star metric is something they call "Happy Deliveries" which is an on-time delivery with no errors. If they can achieve a "Happy Delivery" to the customer then they are more likely to come back, increasing both retention and customer lifetime value.
Let's go back to the example of Spotify, and see if we can build up what their North Star metric might be.
We'll start by asking a very simple question: What does the CEO of Spotify care about?
The answer to that is very simple - revenue. But our North Star shouldn't be revenue, but a driver of revenue.
So, how does revenue translate into a North Star for a product like Spotify?
To answer that question, we must identify how Spotify generates revenue. The company has two primary sources of revenue. One is through ad-supported users, and the other is through premium subscribers.
Which one do you think is larger? Do you think Spotify has more premium subscribers driving revenue, or more revenue from ad-supported users?
90% of Spotify's revenue is from premium subscribers, while 10% of revenue is generated from free users. Not only that, but premium subscribers tend to listen to music on Spotify longer than free users. Data shows that the average listening hours per month for premium subscribers is around 75 hours vs. 25 hrs for ad-supported users.
Given this information, we can guess that Spotify's North Star metric takes into account the number of premium subscribers, because it drives revenue, and the length of time listening to music, because they're in the game of attention.
When we asked Spotify, they said their North Star is time spent by subscribers listening to music.
Is this a good North Star? It measures customer value, aligns with the company's product vision, and is a leading indicator of revenue. It's a great North Star for Spotify.
But we're not done yet. The next step is setting a strategy to make sure we're all rowing in the same direction toward the North Star.
2. Set a B2B SaaS product strategy
The strategy is the path to deliver on the product vision. To create a strategy, you can start by breaking down the North Star into actionable input metrics.
We have found that every single North Star can be broken down into these four metrics:
- Breadth - How many active/returning users are taking this action?
- Depth - What's their depth of engagement?
- Frequency - How often does each user engage?
- Efficiency - How fast do they succeed?
Every North Star is some combination of 3 or 4 of these different drivers. Let's break down Spotify's:
- Breadth - Listeners: # Trial users, # Premium subscribers
- Depth - Content engagement: # Hours per session
- Frequency - Listening frequency: # Sessions per week
If we create a strategy around driving these input metrics, the North Star metric should improve over time.
Let's look at another example from over ten years ago, when Netflix was still in the business of delivering DVDs. It wasn't clear if they were going to be successful. They used the North Star framework to create a compelling experience for their customers.
The first question to ask is what game is Netflix playing? It is definitely the game of attention.
The next question is what is Netflix's North Star? According to Netflix's Head of Product at the time, it was number of subscribers. But they didn't just stop there, they thought about the biggest drivers of the North Star.
Monthly retention was one of the biggest drivers, and they had 92% retention with only 8% churning every quarter. They knew they had to reduce that churn in order to increase their existing customer base.
They broke it down further and found that the number of DVDs in your queue had a huge impact on whether you would retain or churn at the end of the month. Over 60% of their customers had only 3 DVDs in their queue.
Netflix crafted their strategy around increasing the number of customers with at least 3 DVDs in the queue. In the next 18 months, they increased that figure from 60% from 90%, and monthly retention moved from 92% to 98%. That's a 4x reduction in overall churn.
One of the main reasons Netflix has been so successful to date is because of their ability to create a compelling experience that drives the growth of their North Star metric.
Netflix's staggering growth is a perfect example of behavioral science in action.
How behavioral science impacts your strategy
Behavioral science is about identifying the key outcome you are trying to drive. However, this outcome is actually a lagging indicator, which means it takes a long time to see improvement.
Instead, you need to look at the leading indicator of the outcome, and define the input metrics that drive the leading indicator. It will help you set a strategy and align your team around the key input metrics, to create a their sales strategy, content strategy, marketing strategy, product strategy, and so on.
At Amplitude, our North Star is Weekly Learning Users, which is the number of users who get an insight and then share that with two or more people at their company. We break that down into key drivers, and we have pillar goals organized around those.
Our pillar goals are activating orgs, broadcasting learning, and consumption of learning. We need to make it easy for each of these three things to happen.
These pillars identify the leading metrics they drive. For activating an org, we need to know how long it takes for an organization to get their data into Amplitude. If they don't have data, they can't activate, and then we can't grow the number of users.
Over time WLUs will grow if these metrics grow. Which leads us to the final piece, measuring results.
3. Measure results
If you want to grow a product-led company, you must be results-driven. Results show if you are on the right path using an objective measure.
One of the most common ways for a SaaS business to measure results is using OKRs, Objectives and Key Results. In the B2B SaaS world, I've seen OKRs implemented by dozens of companies, and there are some commonalities that aren't actually helping faciliate product-led growth.
OKRs not KROs
Objectives, then Key Results, not the other way around. Vision and strategy are the most important things to set first. Metrics help you measure if you're on the right path, so always start with the objectives first.
This is a stock ticker from a gaming company in 2012-2013. You might have guessed which company this is - it's Zynga. You could argue that Zynga invented product analytics, but unfortunately they took it too far.
They lost sight of the vision - of their Fire Mario - and optimized for metrics. That can create short-term outcomes but won't create a compelling customer experience. And that explains what happened to Zynga.
Design around your learning cadence
Most companies measure all their OKRs for the same time period, the magical quarter. What is so special about a quarter? It doesn't make sense for every company to evaluate all their product experiments in the same cadence.
A quarter feels like a long time. It's 90 days, but then when you take out weekends, holidays, evaluation time, and lost time from other projects going over, you're left with about a month of time to set and measure the OKR before the end of the quarter.
This leads to a repeating pattern of rushed experiments, missed OKRs, and endless frustration.
Instead, ditch the quarter. Think about the long-term outcome you want to drive and how long it would take to get a massive improvement in your North Star. Give your team time to think big about how they can get those results.
Next, what is your learning cadence - how often will you shift to ensure you're still on the right path? A startup might shift every week, and set the input metric for a week. If you're not hitting that metric after a week, go back to the drawing board and try again.
At Amplitude, we recommend a learning cadence of at least 2 weeks, but let the team set their own. Some projects might move faster, and bigger projects need to have a longer learning cadence.
Benchmark for the game you're playing
Always have your North Star metric in mind and benchmark your goals for the game you are playing, whether it's an attention, transaction, or productivity game.
A great example is Facebook.
Facebook's key metric was a frequency metric, DAU/MAU, which is the percentage of monthly active users that are using the product every day. Facebook hovered around 60%, a very impressive measure, and VCs used DAU/MAU as the metric that determined whether they would fund ventures.
It doesn't make sense to use DAU/MAU for most businesses, or even as a good reflection of frequency. DAU/MAU is great for an attention based product. However, in the productivity space, with a B2B SaaS like Amplitude, I don't want users logged in 30 days out of the month because that means they're not being effective with their time and getting the benefit of our product.
Looking at the game of conversion, if I told you a company had a DAU/MAU ratio of 30% you might not think that was very good - it's only half of what Facebook had. But for a company like Amazon, where on their mobile product, 30% of the people who visit monthly shop daily, that is an incredible metric for the game of conversion.
The Product Charter is the single document where you can direct people who want to understand your vision, strategy, and results. You can begin by clarifying your vision:
- Vision statement
- Your ideal customer (whose behavior will change because of this work?)
- Problem (what is their current problem?)
- Outcome (what the customer can accomplish)
- Risk (why you believe you can solve this problem)
- North Star Metric (what is the long-term impact of solving this problem?)
Next, take people through your product strategy, which involves:
- Strategy (how will you achieve the goal?)
- Risky Assumptions (what assumptions do you need to validate?)
- Non-scope (what are your non-goals?)
- Timeframe (time to see the long-term impact)
- Learning Cadence (how long to get to first learning?)
- Customer Partners/Internal Stakeholders (who has reviewed this charter?)
Finally, you can use your charter to provide an overview of the results:
- Context (current state of the world, as related to problem area)
- Problem (the specific area this effort is focused on addressing)
- Objective (what success looks like at the end of this effort)
- Scope (who will be working on this? How long for?)
- Measure of Success
If you want to grow a successful product-led B2B SaaS, you must set a vision, create an effective strategy, and understand the metrics you need to drive the results you want. Then double down on the winners and cull the losers to build a great customer experience.