Every product wants to be sticky and retain users. If you look at retention curves from most products, you’ll see there’s a critical moment where your users will either love your product or leave it and never come back: the OnBoarding. The only problem is that creating a good OnBoarding experience is hard. Really hard. And everybody has opinions on it: Should you drive users to your product directly? Should you guide them step by step? Should you connect them with a human to help them out?
In this talk, I’ll share some of the pitfalls we made at Auth0 while working on the OnBoarding and how we found a new way of working together between Growth Marketing and Growth Product to drive both revenue and user delight.
Martin Gontovnikas:
Hello everyone, and welcome to my talk. The talk is called, the fight For the best on-boarding. My name is Martin Gontovnikas, but everybody calls me Gonto, and if you want to follow me and increase my ego, you can follow me on Twitter at mgonto.
Martin Gontovnikas:
I always say I'm a software engineer gone wrong because I basically moved into, as a developer, what I call the dark side, and now I'm the VP of Marketing and Growth here at Auth0. Let me tell you just a bit about what Auth0 does, so that you understand a bit more about the talk. What Auth0 does is basically help you authentication of a session to your application. So if you want authentication for your application with username and password or social providers like Twitter, Facebook, LinkedIn, Google, et cetera, or enterprise providers like Active Directory or LDAP or Office 365, you can basically use Auth0 for that. And it's a service that is targeted for applications that are being built by application teams.
Martin Gontovnikas:
With that said, let's start with the talk. If you're here, if you're listening to me and you're coming to this remote conference, that's why I put the quotes into here, you care about Product Led Growth. And if you think about Product Led Growth, what it basically means is that your users will be self discovering and using the product by themselves. And then if the product is really good, you can market into them and then they will eventually starts paying. So according to price [inaudible 00:00:01:35], if you care about Product Led Growth, what you basically care about is retention. And retention is about your users trying the application and then continuing to use. In order for your users to be retained, they first need to be activated. And activation is this time when your user really understands the core value proposition of your app, that's when they say, "Ow, this is why I wanted to use this app".
Martin Gontovnikas:
And if you look at this chart, it basically shows the impact in the bottom line of doing a 1% change into each of these. And if you'll see a change in acquisition only change a 3.32% into the bottom line. A change in retention will be double that. And of course, monetization will be even more because it's about dollars. But in this case, we're going to be talking about retention and activation. So if you think about it, that means that focusing on activation is even more important than focusing in acquiring your users once you start getting a good cadence of users signing up. And what that means of course, is you care about activation, just like I care about activation as well. And this was our retention chart at Auth0 a couple of years ago, before we started focusing on activation and retention.
Martin Gontovnikas:
And the important part, if you follow me, is this part in here. Look at this, there's a huge drop. And this drop means that in one period, in our case is one week, we dropped most of the users because this is the average cohort view of all of our retention lines. After that, they more or less stay. So what that means is that for us, if we focus on this first part and we drop less people, it's very likely that we'll actually going to retain more and of course we'll eventually have more users that are paying. This was our retention graph for non-paying customers. So because of this, we said, "Okay, if retention is really important, in our case to increase retention, the biggest change is going to be in activation." And this is probably true for most or a lot of startups that haven't focused on activation because we think that the product will be very easy to understand to every user.
Martin Gontovnikas:
And that's actually not the case. You sometimes need to guide them and explain them things as they are doing. So, based on looking at these, we were like, Houston, we have a problem. We need to change something so that we can improve our retention rates. And in our case, as I was saying, we have a lot of signups every month, but only a handful of them were activating. Those who were activating, were being retained, meaning they liked the product, but it was more about how do we not drop off those users. So we started thinking, okay, what can we do to help with this? And in our case, it was all about, now it's the time to start working on activation. So this was our first okay, now we understand activation is our focus if we want to be a Product Led Growth company, and that's what we want to be at Auth0.
Martin Gontovnikas:
So we tried our first attempt into this. I'm the VP of Marketing, as I was saying, and one of the teams we have in marketing is the Growth Marketing team. So we basically had the Growth Marketing team to start thinking of an experiment or something that we could try to drive the activation up. This is basically what we thought of, it was a guided tour where we would explain you what Auth0 will do for you to help you with authentication and authorization. And it was just a guide that you saw in the dashboard, you click start demo, we will demo you how it worked and what it did, et cetera, and that's it. Once it finished, you needed to set up Auth0 by yourself. In hint sites, doing just a guide that shows you what to do, and then leaving the user by themselves to then set everything up was probably very bad idea.
Martin Gontovnikas:
But of course at that time we didn't think of that. So this is exactly what we thought of. And it's like, okay, it's going to be a guide, they just follow the steps and eventually they need to basically compute it by themselves. So, I put some drum rolls so that they could give some suspense into what happened with this experiment. But I already said that I think it was a very bad idea. And it was, the results were pretty bad that they were saying that we actually did an AB test of showing no experience and showing this experience to some people. I mean, it didn't make no significant change, and in some cases it was actually worse for the users. Because again, in hint site, those users didn't want the guide, at least not all of them.
Martin Gontovnikas:
So we started to do some qualitative feedback, it is something that I recommend everybody to do. If you have done experiment and it failed, and experiment only fails if you don't learn anything. So for us, it was really important to learn something. So we basically picked some users that went through these experiments and we did an interview with them to understand what didn't work, what worked and why. What we learned is that some users actually did like this hands off approach that we did of like, okay, we'll show you what else figures that's for you, and then eventually you implement it yourself. But the problem was that the majority of users wasn't this case. But some users did want that and that it was important for us to understand. Also, something that we learned is that a lot of users actually landed on the dashboard because they weren't getting the information that they needed. They were looking for information in the websites about something for their specific use case, they couldn't find it and then they signed up to the dashboard thinking, hey, maybe I would find in here.
Martin Gontovnikas:
But then they go to this guide and they didn't want to implement the app, so this wasn't for them. But then we also have some users that actually did want to be guided, but they want it to be guided as they discovered the product by themselves. So we would be giving them an interactive experience where we're helping them out while they're going through this effort of implementing the application. So it wasn't a hands off approach. So this is how we were afraid of these learnings. We were a bit overwhelmed because we had some users that like this did guided hands off approach, some that wanted interactive, some didn't even want to read the dashboard. So we were like, this is so much work.
Martin Gontovnikas:
And at the same time, we actually implemented two experiment, these two experiments took us a quarter. So some time went by while we were implementing this, and one quarter in an exponential growth company is like probably two years in another company, because so much shit happens, while just one quarter. And for us, what happened is that now we actually had two teams that wanted to focus on the on-boarding, it wasn't one team anymore. One of them was the Developer Productivity team. This was a team that we recently created at that time, it was a product team that had basically a product manager, designer, engineering manager, and set of engineers. And their focus is to improve the experience of the developer in dashboard. And as they were doing research, talking to developers and doing discovery, what they learned is that most of the users were not in the beginning, which is what we already knew of basically, which is that they need to work in activation.
Martin Gontovnikas:
And for them, the KPI is the time to activation. So time to having one user logging in through our applications or time for retention, that was their main KPI for this team. And we also has the Growth Marketing team, which basically did the experiment that I showed you before. They belong to the marketing team, their main focus is driving more sales qualified opportunities and revenue from inbound leads. Inbound leads for us are people who either sign up to the product, people who either click in talk to sales, to contact us or people who download some of our white papers. And their KPI is pipeline generated from email leads. However, one of the hypothesis for this team, which I think is very true, is that if we have more sign ups that are activated and then more sign ups are going to be retained, we're going to generate more revenue and more pipeline from them.
Martin Gontovnikas:
So the Growth Marketing team also cared a lot about this. So we were like, okay, we tried one or two things, they didn't work out at all, we were a bit overwhelmed. We now have two teams that want to focus on this, both teams have an angle that is interesting, so what should we do? Should Developer Productivity team focus on the on-boarding? Should the Growth team focus on the onboarding? And we decided which product to try an experiments. What if both teams actually worked together in the on-boarding? What if, instead of actually picking just one team didn't make sense? What if we just picked the two, we pick the Growth Marketing team and the Developer Productivity team from product and we asked them to work together.
Martin Gontovnikas:
We basically set two very simple rules of engagement. The first one is that before any of the two teams decides to try something new, once they've done a mock-up and they have an idea, but before they go to the implementation mode, the two teams will meet. Basically to understand from the two KPIs and the two objectives perspective, if these new experiments on the on-boarding and they want to try, would serve both needs or if not, how do we solve that? And the same thing happened at the end. So before we actually shipped the experiment or the AB test that we wanted to do for the on-boarding, the two teams needed to meet to review each other's work. It was very simple rules of engagement, but it basically made both teams have to work with each other, it was mandatory that they work together. And we believed that maybe by doing this, we could try to improve and hit both of the KPIs.
Martin Gontovnikas:
So now let's go to a second attempt. We thought of a new attempt, we called it, request a meeting. The main idea of this is that as part of the on-boarding, we ask some feeds from the user, like for example, are you a technical person? What is your company size? What are you trying to do and why? And what Growth saw is, what if we add an option to request for a meeting as part of the on-boarding? What if, for these people who went to the dashboard, because they didn't find the information that they were looking for, the ones that we learned in the first attempt. What it for them, basically for all of them actually, we add an option for a request a meeting on the on-boarding, and then they can talk to sales if they want.
Martin Gontovnikas:
So the Developer Productivity team, but just did not to show it to everybody, they were concerned that if a developer came, it would actually hurt activation to contact them with sales, which actually makes a lot of sense. So we ended up deciding to only show the option to request a meeting for corporate emails, meaning not Gmail, not Hotmail et cetera, of a medium plus company for those companies that we actually knew the company size. So what was interesting about this is that we actually got to a very, very good place. Because now we're only showing these options to a reduced set that was actually the set that we thought would like to do a meeting, instead of all of them. If Growth would have done it by themselves, it would have shown it to everybody. But with Dev Prod we made this decision.
Martin Gontovnikas:
And, I can't show you the numbers, but this was a huge success. Basically people weren't alienated, people were still being activated and that was still working out okay. And besides that, we actually generated a lot of pipeline from these sign-ups that didn't want to try the dashboard, they just wanted to talk to sales. And we create a lot of opportunities and actually ARR from these experiments. So the two teams ended up working together and this worked well. So we said, "Okay, let's try one more thing." And again, we're almost [inaudible 00:13:42] to actually show people the option to do a product tour. And it was more of a demo and show you everything that the product can do into one place, so that people who didn't want to set it up themselves, or people who maybe weren't developer, they wanted more of a demo to show to somebody they could just do that.
Martin Gontovnikas:
What Dev Prod was concerned was about, what about if people actually see the product tour they click on the product tour, and then they don't go to the dashboard? That would be a huge dropout and we didn't want that dropout. So this concern was very interesting. And based on this concern, Growth decided to add a new KPI, which is, how many people do we activate a month? Like ease that dropping. And the other one was, how many people are dropping now as part of the on-boarding and not getting to the dashboard? Does that affect how many people activate in the end? So this concern in this case, it didn't change how we set up the experiment, but it did change what the KPI was and how we were going to measure if the experiment was a success or not.
Martin Gontovnikas:
So we added that KPI to see what was the drop off in activation besides the pipeline. So this was good because now we care about what that activation right, and the pipeline together at the same time, thanks to these two teams working together.
Martin Gontovnikas:
And this is how it looked like, basically as part of the on-boarding you will get a binder like this proposing you to view the platform tour of Auth0. And again, I can't share the numbers by this was a huge success as well. We actually had a bunch of opportunities and interestingly enough, we increased activation, it was insane. We not only didn't decrease the activation by adding these steps and actually having less people come to the dashboard, we actually increased it because the people who go to the dashboard really want it to do it. So again, this was a huge success.
Martin Gontovnikas:
So we said, "Okay, what are the things can we do to improve even more the activation?" And the Dev Prod team had an idea to integrate quick starts. What the Dev Prod team wanted to do was a guided experience to integrate Auth0. Growth had already done a guided tour as well and failed miserably. One of the learnings we had was that people wanted to use this guide as they were setting up their own application. So what Growth [inaudible 00:16:07] just did was adding interactive setup steps. So you can interactively set up your Auth0 account while you're being guided about Auth0. And this is how the experiment looked like. You pick what type of application you want to do, you then pick what technology you want that application to do, and then you will get a guide with multiple steps. But look at Allowed Callback URLs and Allowed Logout URLs, you can actually set URLs that will configure your account for real and code that is designed for your account to set up Auth0. This we actually just did last week.
Martin Gontovnikas:
So unfortunately we're still waiting for results. I don't know how it's going to go, but I'm actually very optimistic. I think that this is the best of both worlds, where we're giving them a more minimalistic dashboard experience, but they are setting up their Auth0 at the same time. It's not like they are just being guided and then we just leave them hands off. So we need to wait and see. So as you saw, getting both of the teams working together was a huge success for us. Even though in the beginning, we were a bit scared that they were going to be conflicts and we didn't know if it was going to work. But we are a really experimentation driven company so we said, "Let's try an experiment."
Martin Gontovnikas:
And what we learned is this, healthy conflict is awesome. If both teams actually want the best for Auth0 and they do, and they want the best for our users and our customers, and they do, having healthy conflict is absolutely awesome. Because now this team can discuss and get into a better output by thinking of multiple KPIs that each of the teams think of. And healthy conflict drives them to get to the best solution for our users, because we want, of course, the users that want to use out Auth0 to convert into SQL. We also want to reuse activation. So if we can optimize for both KPIs, it's even better.
Martin Gontovnikas:
And what we've decided is now Growth owns pre dashboard on-boarding. So before you went to the dashboard, you're doing the on-boarding on setting up your account and asking questions about you and stuff like that, Growth owns all of that part of the on-boarding. The Dev Prod team now owns the in dashboard on-boarding. So once you get to the dashboard, they will guide the user inside there. However, both teams can add the things anywhere. Even though the teams own a certain part, that doesn't mean that they cannot make changes to other parts. And this is very important because with the rules of engagement of checking before, and then before starting an idea, and then before shipping, it still works if teams basically can add things anywhere. But we added ownership so the teams can feel ownership as part of the on-boarding as well.
Martin Gontovnikas:
So if you only get one thing out of this talk, is that you should increase healthy conflict because it can help fuel both revenue growth and user happiness. And of course the quote is by being a contemporary unthinker. Thank you very much for your time, I hope you enjoyed the talk. And as I said, if you want to follow me and read some of my ideas, please follow me on Twitter, on mgonto. Thank you.
