In this bonus section, you'll get access to previously recorded live expert workshops and Q&A from PLG experts such as Todd Olson (CEO of Pendo), Tara Robertson (CMO at Teamwork), Nelson Joyce (Co-founder of Tettra), and more.
Wes:
Like always, I like to start off with the good questions. So let's actually pick on Todd for a second. So Todd, can you tell us something about your backgrounds that has brought you to where you are today? What's the background behind how you came to even start Pendo, storytime now?
Todd Olson:
Geez. Well, I'm old. So, that means I have lots of stories. But I go with [inaudible 00:00:30]. I started building software in my teens. I started my first job was actually working for a bank [inaudible 00:00:39]. It's a credit card institution. So it kind of cut my teeth on building software on a combination of mainframes and old Unix systems. But I think it was a great experience in that I learned a lot around going deep with customers. And given that I didn't know any, I wasn't a trained professional, no one taught me how to be a software engineer. I just assumed you're supposed to go talk to your customers all the time and get feedback on how things are going. So I was really customer oriented and as the fun fact, I guess, that this is after a few years of doing it someone finally figured out that the bank's systems in this particular department were depended on a teenager and they all freaked out and started hiring real people around me.
Todd Olson:
But that was just through attrition and a few other things. I was like, "Okay. This person is the only person knows anything about our systems and he's 16, eventually is going to go to college. We're going to lose him." So, but those are really good experience, definitely helped me be the person I am today.
Wes:
Yeah. Now, that's has been a fun realization for that bang, like really?
Todd Olson:
Cool. So what happens I think I went to Disney World with my parents for a week-long vacation and I got back and they gave me a massive raise and because it probably was a lot of things did not work, [crosstalk 00:02:07].
Wes:
Todd.
Todd Olson:
They were like, "Oh, this is a really dangerous thing." So it was pretty fun.
Wes:
Awesome. Well, thanks for sharing a little bit more about your background and how you came to be where you are now, because it's always interesting to hear how we all started with, whether it's how we got into tech or anything else. Because that's how it all started and it's interesting in the past that it takes us at the end of the day. So I'm not going to wait any longer because we got a really interesting topic today and just around how to build a successful product-led business. So as I mentioned in the beginning of this, Todd recently wrote the book on The Product-Led Organization. I have it here, I have read it myself and what is really trying to tackle here is just, what are all the inner workings of a product-led organization? What is different about this? What are some of the things that even when it comes to the measurement? What do you need to do differently?
Wes:
And so I definitely believe that the whole approach of building a product-led business, there is every a team has to work differently, has to measure things differently. And so I'm really excited to have Todd here to really just break down this whole concept of a product-led organization and what is different. And so if you have questions as Todd's going through his presentation, make sure to just put it in the chat and I can pop in there and interrupt him as nicely as I can and I can anyway, and then we can make sure that we dig into some of these concepts too, but it's going to be a lot of fun and I'm excited for it. So Todd, if you want to share your screen, you now have access to do that.
Todd Olson:
Got it. Let's do that. We will jump in. Thank you West. Awesome. Well, greetings everyone. It's great to be here. I'm going to share this talk with you and again, as West said, feel free to throw your questions in there, happy to make this as interactive as possible. Certainly do not want to put anyone to sleep so that would not be my goal. So love to engage. And I'm going to... Today's talk is really... Want to focus on an area of the book that actually... The third section of book, which talks about A new way of delivering products. I'm going to start off by talking about it all the way to delivering product of course, which is and heck some of you may not even have seen these, this is DVD in my Mac that I'm currently showing this, but didn't have a CD slot anymore. So I thought obviously it gives you a sense for how obsolete these are but this obviously used to be the way we deliver product. I shared a very real story. One point in my career, I worked for a company that actually shipped boxes and absolutely that shipping of a box affected a large way about our motion and that ended up being of course, the constraint in our ability to get feedback. It was our constraint and our ability to engage with customers in this physical medium.
Todd Olson:
And one of the exciting things about today in cloud-based businesses is that obviously we have this reach that we've never had before, but and that's opened up new opportunities. And I think that's one of the driving factors which in the book I talk about as creative as opportunity to be product-led, right? I think if we were all back shipping CDs, I think it'd be really much more challenging to be product-led, but now if you look forward today, I think we have a lot of conditions by which this is now very much a possibility. So what do I mean by product-led organization?
Todd Olson:
And I think various definitions of it. I think West probably got his own definitions is as well, but what I like to talk about is putting is this evolution from product being a part of the customer experience to being central to the customer experience. And if you think about it, every role, every department has an opportunity to leverage the product to perform its function better. And of course, we always know about sales and marketing has the most of the traditional product-led growth areas that leveraged product, but it could be finance for collections.
Todd Olson:
It could be recruiting. I think there's actually talked to a couple of people teams that use the product to find new candidates still work at the company. So there's every aspect of the business can leverage the product to perform, throw more effectively. So and then by doing that, by putting product the center, it also then creates this I'd say requirement almost to rethink how you deliver that product. So once you do that the way, you deliver your practices, your processes even your organization should evolve to meet this new demand. And one of the examples, I like it because it's an OG examples. Etsy, you go back several years when Etsy was coming out of the scene where things that we're very well known for is this whole like very rapid release cycle where they're... I forget now, certainly multiple times per day, but it was primitive times per hour.
Todd Olson:
And then there is this very elaborate measurement system where if they shipped a release that hasn't had some negative effect on their business, it would automatically roll back. And I remember it being one of the first real examples of continuous release that was really highly publicized. And by doing that, allowed them to iterate really fast. It allowed them to increase quality and again, this is a shift in way they delivered their products to again, be more product-led and it enables speed. When you have elaborate systems like that, when you have those controls and mechanisms, the outcome, the benefit of that of course, is just greater speeding and greater speed while maintaining high quality. I think that's one of the really cool things about that example is that they were able to go really, really fast and maintain high quality.
Todd Olson:
So the first thing I really want to talk about is thinking beyond the launch and I start there because so often in product, everything is about the launch. And then it's kind of like, "Oh, it's pop open a beer and then celebrate this launch and like move on to the next thing." But the reality is, the launch is actually when things start getting interesting and I coach our product teams too, is like the shipping is not the celebration event, the celebration event is when you're actually delivering value, when the customer actually recognizes, right? And so I want to talk a little about different ways to rethink shipping mean but as I started, I talked about CDs. I talked about, I mean, we used to build things for way too long and then we would ship it and literally physically ship it.
Todd Olson:
Even we went first to cloud software, I found that our cloud delivery cycles matched the physical mediums because we're so used to building, but now you're just have so many more opportunities to and Etsy was an extreme owners multiple times per day, but you can rethink how you get software in customer's hands. And this is... I talk a lot about this in the book, but the advent of feature flags as really opened up all sorts of interesting possibilities. And if you should know what feature flags are, if you don't. The ability to get features either for customers or for specific users. So different feature for new systems have different elaborate schemes, but as a product manager it opens up an incredible number of possibilities for how you think about shipment.
Todd Olson:
And this is a standard example release schedule, but you may have an internal phase. Internal meaning your employees can use your software first. This is obviously very, very useful if you're a company that can use its own software. There's a [inaudible 00:10:36] like the term drinking your own champagne or eating your own dog food, but this is a really great way to get it in friendly's hands and hone it. And obviously your employees are a great source of feedback if they are in using your products, right? It's also nice, it's kind of a hack almost for enablement. In almost everything you ship, you need to teach people that your company will support or sales, what it is that you're actually going to be supporting and or selling and if you have an internal release? It's actually a really, really great way to have them experience it and ask questions about it prior to customers getting their hands on it.
Todd Olson:
Then I kind of highlight this notion of a limited beta and a limited beta can be limited in any form. It really, really depends on what you're trying to accomplish. I've seen limited betas that are just... We're all control groups or there are percentages, it could be a random algorithm that picks limited. You can't pick limited based on maybe you're really concerned about this feature from a performance perspective, and you want to pick some outliers in your account base that could really test the system because you want to make sure you get it right before your shipping. Maybe it's a friendly and you know this customer is a power user and this person will give you amazing feedback if you get it in their hands first and also be a little bit more forgiving. I've talked to some customers around, they charge for this, they charge extra money for a customer to test your software for you.
Todd Olson:
That's kind of a crazy idea, but I've seen this happen, why? Because it is a value to the customer to co-create the outcome. Everyone wants to build things and if I as a customer, get the special invitation helps shape the future of the software that I'm using, maybe I would pay more for that. That's pretty cool, right? So the limited betas are really, really powerful. There are ways to and how you use it again, probably depends on the future and when you're trying to validate, but it's all about validation. I'm trying to validate some learning. I'm trying to hone this prior to opening up on a wider audience because open beta is opening up to a much wider audience and this can last as long as you want.
Todd Olson:
Usually when we label something as beta, we're trying to communicate that it's not done, that's what we're trying to say. "Hey, here's the thing, please play with it. We don't think it's finished yet." Now in the case of Gmail, if you recall that it was beta for years. And that was maybe an extreme example and I think it was almost... It became less of a joke for them at some point, but I think they legitimately for a while probably felt like it just wasn't quite done. So that's hence why they need the beta.
Wes:
And I might just add to this as well because everyone here is going through the transition from the sales-led to product-lead. And so when you think about even as this might apply to launching a free trial or something new, I think this whole approach is fantastic just because you can definitely get a ton of feedback internally, but then have that limited beta open it up a bit more before really showing it to everyone. And so there is that feedback loop at every stage you're learning and when you think of the internal processes involved with everything that goes on with launching a free version you can build that muscle slowly and then by the time you get it to opening up for the public, there's a good amount of muscle there internally as a team. So I think this way is definitely fantastic to really reduce your risk and learn from it at each of those stages. So the question I wanted to ask you is how do you know when it's time to progress to the next stage? So let's say we do the internal launch, when does it make sense to go to the limited beta and then the open beta?
Todd Olson:
Yeah. Great question. This is an area where I really recommend teams set criteria in advance. We will promote from internal to limited once we hit this given milestone and then I'm usually that milestone is some learning event. We really care, we don't know this, so we're going to put it out there and get feedback to help answer these unknown questions. Once we answer those unknown questions, then we'll promote to the next stage in the hopes that we get graduated learning so that we add new learnings at each phase. The other thing that I've done in one instance, we've actually done surveys, when we do a simple Likert scale, how did this meet your needs? And we're getting more fours and fives than we did ones and twos.
Todd Olson:
And we're like, "Yeah. It feels pretty good." Right? So you can do some for a cohort analysis and look at the newer users to see if we're giving you higher scores. I've used that as well. So but it really depends on what you're trying to learn. So I got to just some performance thing. I wouldn't promote it until I've tested the performance, then validates and checks the box and is good enough and then once it is like, "Okay. We're good on that." It's going to get promoted to the next phase. We've had some, even at Pendo, every once in a while we release a feature that you just don't know how it's going to work and until you put on live customer sites. And anything like that, we have pretty heavy betas, because we think it's going to work really, really well on my customer sites, which don't really know until you throw it on there and so that's a great opportunity for beta.
Wes:
And Gareth had a question around just like, do you use MPS or like some sort of PMF survey to really understands your users at each of those stages to move on?
Todd Olson:
Yeah. I mean, I probably wouldn't do an MPS for advancing a feature. We do more micro surveys, something a little bit lighter. I mean, I still think MPS is probably a better use just overall sentiment you have towards either the product or the company and even the PMF survey is probably trying to. That may be a decent one to use, I think it's still probably a little bit heavy. There's for this usually it's just like how does this meet your needs or like a basic Liker scale, one to five is what I've been using. But you can use whatever you feel does it again. MPS, I think is a very different question. I probably wouldn't use that.
Wes:
Yeah. And Gareth, if you want to dig into a lot more different ways of testing, this book, Product-Led Organization, honestly it's audio not so like so many I was like, "Oh my goodness. Did you know there was half the ways that you could really measure this." So it's fantastic.
Todd Olson:
Yeah. I like survey methodologies. They make me happy. So I covered a bunch of them. So cool. This actually next example, is it actually decent testing? Is this a form of testing? This is one of my favorite techniques. I call it the opt-out beta, not opt-in beta. And the opt-out beta is, you release something new and this is an example, this is particularly useful when you're releasing a new screen for example, a new version of something. And but then you give the users the ability to go back to the old one. So you kind of like shock them like, "Oh wow, here's something new." And what you're measuring is how many people decided to go back to the other one? How many are willing to take the... What's the matrix where you take the red pill? I can't remember which one's the good one or the bad one now. Absolutely [crosstalk 00:18:05]. But you get this, you get the point. So and this is really, really good because if you had opt-in beta, people would just fear change.
Todd Olson:
That psychologically is going to block some of your testing. And so in some ways it's not the best test, right? Because if you're just asking someone if, "Hey, do you want to flip the change on you right now?" A lot of people are just going to say no for the sake of saying no, but if you throw them on a new experience, now it's changed from the go back. So then the question is, is this really so much worse than the prior experiences that people are willing to change to go back much? I found much more effective tests and this is one where you get your basic cohort analysis. I've done this in a way where you can see the first couple of weeks, a lot of people go back then as you improve the product. And of course, when someone goes back you ask them, why? Why did you want to go back? And you then variable, you'll hear like, "You got rid of this." And then you're like, "Oh yes, we did get rid of that. But we were hoping it didn't matter."
Todd Olson:
So I always think these are great too, because you can experiment how much someone really uses something by immediately taking it away. You still get them to go back. So they're not going to be pissed off at you because you give them an out, but it's a great way to test if something is actually used. So it was a little bit of qualitative feedback. This is a really, really, really effective technique and again, it's more effective for basically the entire screen, like rewrites of pages, but one of my favorite techniques here. So also when it comes to post release, the other just basic stuff is focused on driving adoption of it. This is example of course, I've told tip and you can see we're highlighting the ability to use this feature.
Todd Olson:
This is just good hygiene when you're getting something out there, tell them about it, but the buttons new or it's moved or it's... And in here, this is a good example of what... If you have the real estate for it, try to even tell the customer why. Why should they go to your new [inaudible 00:20:11]? Why should they be adding new elements? What did they get? What value do you think like, "Hey, you're asking the user to do some work, what value they're going to get for doing that work?" That's a great thing to be thinking about when you're trying to drive adoption. So this is one of my favorite topics, the art of letting go or in this case, I'm going to talk specifically about rewrites.
Todd Olson:
In the book. I also talk about sunsetting features or some people call them sunset that's the fancy or the euphemistic term for deprecating features I suppose or just removing them. So I cover about lot of examples there in the book, and I'm a big fan of sunsetting again, the very euphemistic word for getting rid of things that shouldn't be there. But this is specifically around different ways to view the rewrite. And if you've been in software for any period of time, you've had to rewrite things and rewrites are tough, really, really tough and in the book I highlight this amazing medium post that kind of does a deep dive on them. I'm going to highlight a few of the examples. There's even more examples there, but there's a lot of different strategies you can take with a rewrite. Basecamp, if you know the Basecamp folks, they're now also the folks behind Hey.com, a very popular email client arguably very much a product-led business.
Todd Olson:
They have no salespeople I think so, they're 100% product-led. But they're pretty famous so that they talk about this is that they have rewritten and come up with new versions of Basecamp, but they've always kept the existing products around. And their thesis is simple. "Hey, are all your products rights value?" People pay for it. They like it. Why do we force them to move to something new, just because we wanted to either clean up tech debt or we had a new vision for it. So they have basically through all of their leases, I think they've done it a few times now, put new versions out and that's okay and they still charged you for the old version, they charge for the new versions. For them, it's just part of the evolution of their products.
Todd Olson:
And so for them, they maintain multiple code bases, et cetera. And I think a lot of people don't want to maintain multiple code bases, but they're like, "Hey, people are paying for this. It's a good product. It works. Why would we get rid of it?" So, that's actually one interesting strategy. Their FreshBooks one, we happen to know this example pretty close because we were working with him on this strategy. What they did was they were a little bit slime at the whole thing, they built a competitor to their own product and they call it a Bill Spring. They actually put it under a whole different URL. I think they even maybe incorporated it as a separate business. They went over the top for you to think that it was just a straight up competitor to FreshBooks and they just ran it as a test.
Todd Olson:
And what they say is they knew they were successful was when they were getting win-loss reports at FreshBooks for people going to this other product called Bill Spring. And they were losing business to the new product they put out there as their own competitor. And once that happened enough, they knew they'd built in their mind, a better product. They then kind of, "Hey, you caught us, here's..." They kind of exposed the whole ruse and then eventually kind of merge everything together and find upgrade paths, et cetera. But a really interesting way to do a test, and because by doing it as so secretive, you eliminate bias from the test and just came down to which product is better. And the new one eventually became better than the legacy one and this is pretty cool for us.
Todd Olson:
We were partners on Bill Spring since day one. So we got to see the growth because we collect analytics on software products. So anyway, very really interesting one. The third one, this is Inbox in Gmail inbox. If you know Inbox, it was an app for your phone. This is really, really interesting. So they put this thing called Inbox, it was just like a different UI essentially on top of Gmail, but they drove it as its own app. And over time, what happened to that is they ended up just killing Inbox and what they did was they took all the interesting features in Inbox that the people were using and they made them part of Gmail. And this is a really cool example because they saw this new app is just an experimentation platform for new features they wanted in Gmail. They knew if they put it in Gmail where they did have lots of bias in it, people would get pissed off because it's changed. So they're able to... People self selected it, "Who wants to test out a new product?" Well, people that download the app say, "Oh, I do, I do, I do." So they got to hone it and eventually it replaces the existing one once they have a chance to really hone that. So three actually very different examples. Three interesting ways to think about rewrites, upgrades, letting go of the past.
Wes:
Yeah. And I'll jump in here too. So as it relates to make that transition from sales at a product-led, why I find this so interesting especially with the FreshBooks example is when you think about changing your culture, going from sales-led to product-led, sometimes having a small tiger team work on something different and really like in this story too, they did siloed them, they were, I don't believe they were even in the same office, they got rid of them, sent them on this project. Same thing with the startup clothes.com, they were initially just focused on doing inside sales for a bunch of companies and then Stelly had to make sure like, "Okay. You have a few months work on this." And to get them rolling. And so sometimes that can be really effective to make a ton of progress really quickly when you do that approach. But yes, Jen. I'm still sad that Inbox is gone as well. And is there any question just around the Basecamp? So how do you reconcile the pricing strategy? So for a lot of PLG companies, the best practice is to not grandfather users into their old pricing when you push new features and new pricing. So how do you approach pricing in that particular instance where they're launching new revisions of this? There's V1 and V2 and they're completely different products potentially? How would you approach that from pricing?
Todd Olson:
What principle you're trying to accomplish? Right. What you're trying to accomplish and usually and PLG, you're trying to drive growth through your existing base. So you want to give people reasons to typically upgrade to the new version. So you're putting some new features, some new capability, something in that new product that causes people to upgrade it and buy the new one. And this is actually going back to the CDs. This is what we used to do back in the CD days. When I shipped the product and you bought it off the shelf, the only way I make more revenue in growth is if I put out some new version, some new feature, the new thing that causes you want to buy a thing. So precess, we had no other recourse, but to try to put cool things and the new versions. So I think that was a lot of the drivers of revenue so and so.
Wes:
Awesome. Thanks for [inaudible 00:27:41] on that.
Todd Olson:
Yeah. So, that's kind of different way of thinking about the release. Let's say I want to talk a little bit about is how to create more engaged customers. And this is a slide I've used a few times in the past, but I always say once you ship something, people are going to have an opinion about it. You can't stop people's opinions, right? Now, the challenge is how do you manage all these opinions and feedback? And if you don't create some sort of system end of process for it, you have a possibility of ending up with some sort of feedback chaos where you have little bits of feedback and Slack or support tickets or phone conversations and it invariably then you end up with these different spreadsheets.
Todd Olson:
Like one of my pet peeves walking into an organization is when you have sales top 10 feedbacks, success top 10, customer ended up 12 top 10 lists, which it's impossible to prioritize 12 top 10 list. So what you want to try to find a way is to create some centralization, centralized repository, centralized voting schemes, but some way to manage all this feedback in one place. And if you create a single hub for it, people will use it. If you don't create a single hub, you're going to end up with this chaos because people will just send it in a whole bunch of different places. There's often in your email, which is the worst place for any sort of real feedback. The other nice thing about centralizing it is it creates an opportunity for a two-way constructive conversation.
Todd Olson:
Because if someone asks for something and you don't know exactly who it is, you can't go back and ask, "Why did you want this or what pain are you trying to solve or how would you value this work for me?" So having it centralized just knowing contact information, you can actually create that two-way street. Then if you would build it, you can then tell that person that requested it, "Hey, we listened to you. We delivered this thing." As a great way to build loyalty and have a happy user for life. So that because everyone likes it as I said earlier, being part of the creation process. So this helps you empower you to have your customers be part of the creation process. And when doing that one of the things to consider is how we do prioritizations? How we do voting?
Todd Olson:
There's a lot of different schemes, I know just a few. First one is called where I'm going at basic voting and it's just like, "Hey, you want this thing?" And or "I don't want this thing." Now, the challenge is in my experience is that if you let customers ask for everything, they're just going to want everything, and you will end up with 100 top 10 features. Or you can just say, "Well, you know this customer you can do it then by size, you can pair other metadata with it." But this is a very basic scheme for just getting a sense of what's most important. The second one, which I've used quite a bit basically it's a budget style voting, which is, let's say I give a customer or user, even internal stakeholder, 100 points for lack of a better word or better term.
Todd Olson:
And they can apply those points across any feature they want. They can put all 100 on one, if they really want to or they could smatter the costs. Now what you're doing by giving them a budget is you're forcing the customers to make trade-offs, which is exactly what you have to do when I'm building a product you have to make trade-offs whereby... So then they are sharing the pain, so to speak with you and you're going to get very often better answers because everything can't be a number one, and that's one technique. Then the last technique, which usually requires some sort of system, because it's a little bit harder to do is it presents the user with just two items and the successive number of two items and allows you to say which one's more important to you.
Todd Olson:
This is also a really nice technique because it only requires the user to say, "When I'm looking at a versus B, which ones have more value to me?" And that's a pretty easy decision than say looking at a list of 100 things and trying to figure out what's number one of 100 things. This is a really powerful technique if you do have large numbers of items that you want strong prioritization by giving people successive questions, A versus B, B versus C, et cetera. You end up getting a pretty good ranking just based on that user's needs and interests. So those are the three schemes, they're all better than nothing. So if you do nothing all for you, these are better. But there's definitely pros and cons of each of them. So the other thing I think is really, really important is whatever your strategy is, publish it, share it.
Todd Olson:
This is one of our customers Kajabi, they had this cool video, in the video they talk about what they want from you when you give them feedback. They're like, "Hey customer, we really like you to tell us the pain, not just what you want." That's good advice. "So we would like you to tell us there is a..." So be specific, tell users how to give good feedback. Then it's also a good opportunity to tell them what to expect. Hopefully, you're going to communicate and acknowledge the person's feedback. So how will you create two-way communications? Then of course, how to get started? Where to go do it? How to get updates on it? This is really, really, really valuable. The last thing you want to do is if you're asking customers for feedback, is it for it to be a black hole?
Todd Olson:
And just the responsiveness itself will engender loyalty from customers and one of the other little tricks I've seen throughout the years that I really loved was, Elastin did this a few years ago, is when you submitted a feedback request, they didn't just say, "Hey, thank you for the feedback request." They did, they thanked you, but in that email, they also told you what was on their current roadmap. So it's like, "Hey, thanks for asking for this, by the way, this is what we're working on now." And what's cool about that is that a lot of times you got that email and they're like, "Oh wow. I really actually wanted those things where again, now more than the thing I've just asked for." And so it's a really good way to reinforce what's going on now. You'll get a lot more points from it.
Todd Olson:
So and then of course, once you have all this great data, you use it. Build what people are asking for, I mean, this isn't rocket science I guess, but this is actually an example from our literally usage of this sort of capability. And you can see that you have delivered to GA, we're drafting. Some of the things were planned, some things are already in beta. We now actually have Pendo, we're big enough company where we have a team that all they do is pull item. They create backlogs based than just direct voice of the customer. The customers essentially direct almost an engineering team here. And it's really, really... The good news is that constant customer wins. And it's a great way again, increase loyalty with your customer base.
Wes:
And when it comes to taking all this feedback in I'll share a contrarian view of this. So we were interviewing the CEO of Better Proposals and whenever it came to customer feedback, he's like, "Always say no." Initially I was like, "Yeah. I don't think that's really the best approach." But then he kind of like backed it up and said like, "Well, for the first time you hear something always just say no. And then when you start hearing things again and again and again, that's when you start paying attention." So I'm curious to hear your thoughts just around, how do you balance what the customers are asking for and what is the overall vision for the business and where you want to go as a business in that overall direction?
Todd Olson:
Yeah. I mean well, I said one team on it not 12. So it's obviously a balancing act and this is part of whatever strategy you want. And this is again, this goes back to the first chapter in the book, I think talk to starting with the end in mind. And so to knowing your goals, if you're having issues with net promoter score and it's something you want to increase, maybe spending a couple of quarters listening to your customers and what they really, really want is a good thing. I would never have a strategy where all I do is build what customers say and have no vision for the future, because if you do that you're not anticipating what customers need, you're going to get behind, a year from now you have lost innovation. It's ultimately going to be a really, really bad thing. So you always have to be building for the future and for your strategy, but taking curvy off some capacity for something like this is really, really, really healthy. So it's a blended prioritization model. Is there a right answer? This depends on your business.
Wes:
Yep. Absolutely.
Todd Olson:
The other big thing is roadmaps are a communication tool and I think a very, very effective one. There's a lot of [FID 00:37:28] out there about roadmaps, don't show roadmaps or now I'll caveat this, if you're a product manager don't put a roadmap out that your engineering team has not seen, that'll just them off. So then they'll feel like you're making a commitment to the market that has not been validated by them. So don't do that, that's bad. But use it as a way to get feedback, one of the best ways to get feedback is to throw something out there and saying, "This is what we're doing." And say, "How does this align with your needs?" And people will be pretty honest is what my experience says and then you can do roadmaps in various ways.
Todd Olson:
You can curate it, you can put it live on the internet. Plenty of companies like Trello certainly used to, they probably still do, so but plenty of companies do this. It's a great way to engage with your users. And again, build this loyalty and people when you have your users to a point where they want to spend their precious time helping influence your product roadmap and what you're building next, you kind of have them hooked, right? Isn't that like Nirvana? When they cared that much to be active participants in the future of your product and business, that's like the greatest thing in the world. How would you not want to want that? So this is a great opportunity. Now the last piece I want to talk a little bit about is what I'm calling the orchestrator product operations or product ops or sometimes it's called prod ops.
Todd Olson:
But a lot of these new techniques I think have necessitated the creation of a new role. And the way I think about that role is it kind of sits at this intersection of product success engineering. So it's a good healthy cross-functional or that like some people will probably say, "Well, doesn't this product also cross?" Yes, of course, product is this is but I see product ops as being again, a real orchestrator between these various departments. And that the power here is what product ops is now doing, typically which is following product managers' shoulders. And I've said many times that product managers already have one of the hardest jobs in the business. They have to be internal and work really, really heavy with engineers and write specs and accept stories and hone things.
Todd Olson:
They have to be external. They have to be talking to customers in the market and et cetera. And I didn't know all the stuff like, "Oh, they have to manage all the release cycles and manage all the customer feedback and they have to make sure our data is consistent." Well, that's just more stuff taking away from talking to customers and work with engineers. And I want them talking to customers work with engineers, that's where the magic happens. So, think of product ops is taking, relieving the product of some of these responsibilities that they have to do. And look, I mean, what they do again, complete dependence on your org. Here's just an example template we threw out for regular meeting, but they look it if you do have a sales lead motion or even if in your system, they look at requests from a sales or marketing team, they look at customer success input as well, which could be requests.
Todd Olson:
If customers have asked for some sort of commitment for something, they manage those commits, they review feedback, roadmaps, competitive intelligence, they manage the beta cycles. If you're running these control roll-outs, there can be chaos if each product manager is running their own... What you don't want is your product to be just full of random betas, you want some method to the madness and this team can manage that method and manage the launch process as well. And then if you are doing something like a Net Promoter Score, they can review the scores or the qualitative feedback and make sure that your company understands what the sentiment is at any given time.
Wes:
And the so Kyle just had a question around this. So they are mostly responsible for product discovery, what is the main responsibility would you say for product ops?
Todd Olson:
I wouldn't say discovery. Discovery, I'm still expecting it product managers to probably do. I think it's a lot of the things you're seeing, working with the customer success teams working on anything that goes across more than one product manager, this team's going down. So they're trying to create more consistency and bug process is another great example. Almost every company has some sort of bug process and you could just take turns between the various product managers going to the meeting, or do you have a single group that's owning it sits on top and there's an divvied out across all the different teams.
Wes:
And this one's my question, but when do you think is the right time to introduce product ops?
Todd Olson:
Yeah, I mean, there's no question there probably needs to be some level of maturity and size, but once you get above four or five product managers, I think you're probably ready.
Wes:
Right. Okay. And then Jen just had a question regarding this, how many teams could a product ops person cover? And you mentioned in that case, there's four to five product managers. Is that kind of the typical or if you've seen it work more effectively, if it's bigger than that?
Todd Olson:
Yeah. I think three, four or five teams per product ops person feels about right.
Wes:
Okay. Awesome.
Todd Olson:
The other big thing that we've seen in product ops roles is managing the tool stack, I mean, interestingly, there used to be very little tools for product it's kind of PowerPoint and Excel and well now there's a whole market map like this of all the various solutions you can use on which Pendo of course, one but there's lots of them and someone's got to manage it. Someone's got to set it up. Someone's got to organize it and that itself is a job and this role is what we often see there. So, look I've talked about putting product the center of the customer experience and how that influences how you deliver and I think the challenge I have for everyone is, how can you evolve your org to help support your product-led motion?
Todd Olson:
And if you don't do it, you run the risk of the org driving you and that's always the question to me, I like to anticipate change and get ahead of it so I can control it and drive it myself. So I would encourage you to think about ways you can change the way you're doing things to support more product-led motions. And these are just some of the customers that we've been working through. And I think one of the exciting things is that they have really evolved different aspects of their business. So I know that like the Jamf team for example, has done a lot of work between UX and product and trying to simplify a lot of the workflows in their product to drive more adoption of certain areas in very product-led fashion. Rapid7, similar examples of over their UX team. So if you do this also fun fact, a lot of these people that have led these product-led initiatives of these businesses have all been promoted. So this is a great just for all of you that on this phone call, this is an opportunity to get visibility inside your organization and you will be recognized. So that to answer more questions and take it where you guys want to take it.
Wes:
Awesome. Thank you so much for going through that presentation. That was awesome. So for everyone else here feel free to just put your questions in the chat and we can go through them based on just the order. The first one I'll answer. So for Claudia, regarding the book above signed up, but was neither prompted for a download nor did I receive an email with a link. So I did check with the vendor team, they are getting your contact info, I guess there's just some little bug on the form. So don't worry about it. If you fill it out, you will get the book and they are checking on their ends, but it did seem a little buggy. So sorry about that.
Todd Olson:
Sorry about that. Oops. Wow.
Wes:
Oh good. And so one of the things I was talking with... Actually before we started this was just around the whole balance of how do you decide what do you give away for free versus what should you charge for whenever you're launching a free plan? Because it's really tricky, you give away way too much for free, you might not have a good business to run. You go and just over optimize and really getting all the features, you might just kill your customer acquisition model in the process because no one wants to sign up for that product. It's not exciting when you get in, you don't get anything fun, no goodie bag for you. So how did you structure that with Pendo, it's recent launch the free plan?
Todd Olson:
Yeah. I mean, there's [inaudible 00:46:41] website, I think I got to find it. I'll try it. It's something that's like enterprise.com, I think it was an interesting website, because it kind of talked about what we often see people charge for all the enterprise features, in the summer it's as basic as security stuff that people are willing to pay for better security. But I think look, the key thing you needed to prioritize in the free product is getting people hooked. First and foremost needs to be a good product. If it's cripple where, if it's feels like it's arbitrarily crippled just to drive some sort of PQL conversion, it's going to fail. So you've got to give away something that of value, it's got to work.
Todd Olson:
So every product has a core reason people buy it, then he's going to be [inaudible 00:47:37] in my mind. It can't be some hokey side thing that is only a small part of it. It's got to be the core of it, or it's not going to work. From there you've got to figure out ways that... What would people pay for? People pay for security and privacy. People will pay for potentially better service. People will pay for integrations potentially. Does integrations require you to maintain them? That's a little bit of work usually. So I think there's a variety of things that you can do above and beyond the core functionality to monetize. That's kind of how we take it to the integrations. Anything advanced is kind of how we put it, but the core product works, it's great. If it does something for free, that's really, really valuable and we're having high usage of, for that reason I think you have to focus on that, otherwise it won't work.
Wes:
Yeah. I'm definitely with you on that. I think to kind of sum that up to a good rule of thumb whenever it comes to, what do I give away for free versus what to get? Is you need to ask your users, would this be valuable enough for you to continue using on an ongoing basis without paying? If you can't answer that and they can say like, "This is absolutely valuable." Then yes, you're on that line of like, "Where's the balance between what you're giving away and what you're getting." So it is a fun balance game for sure.
Todd Olson:
Yeah. At the end post the enterprise ready, I think this is great, I mean, I've spent so much time on that website. And you're right like in this website, it's things like single sign-on. This is a great example too when I'm in companies I talk about like, "We charge for Salesforce integration." And people say, "Why do you charge for Salesforce integration on if you'll have Salesforce?" Yeah. I'm like, "How much is Salesforce cost?" It's 100 bucks a user like, "Yes, if you're willing to pay that, you probably wouldn't pay me something for it." Right? So that's always a good role, if you're willing to have a single sign-on provider like Okta yes, you'll probably pay me to support that single sign-on provider. If you're not then no, I'm not going to... That's a great, I mean, so that would never be in free for us, right? And there's these heavy costs associated with it, but it's a great way, to me I always think of it as well, so.
Wes:
Awesome. And so yesterday I was asking everyone in this group, just like what are some of the biggest challenges around building a product-led business? And what kept popping up was just, how do you get alignment within all your teams? How do you get buy-in for this product-led initiative and how do you start moving this ship to this specific direction? So as the leader of Pendo as are going through this transition, I want to hear just like how have you really helped everyone get bought into this vision and start moving in the right direction?
Todd Olson:
Yeah. Well, I covered this a little bit earlier, but we have a company planning process like OKRs, not OKRs but something very, very similar. We have a three to five annual priorities and then each quarter we have kind of a roadmap, how we execute against these priorities. And for something to be a priority, it's required to be cross-functional, it's something we can accomplish of course, within a time box like a year end or a quarter. And once it gets that it's going to be heavily communicated, we'll communicate at every town hall. We do quarterly retros. We print them on giant posters and now people aren't coming to our office. So the posters I guess, aren't as the most useful thing. [crosstalk 00:51:26]. Yeah. So but you got to constantly communicating, communicate the why and give people updates on it.
Todd Olson:
So we've been very, very communicative of all of our efforts, free initiative. We've given lots of demos of it to the company. Every major milestone gets celebrated and that's been a big part of getting the company fired up about it and the company is fired up about it, but they're hearing about it a lot. For a 400 person company, this is one of the three most important things we are doing this entire year. So you're going to hear about it a lot and I think by doing that it also... I talked to you earlier about, I mean, it was before this call about this all the ops stuff.
Wes:
Yeah.
Todd Olson:
And we had to get the sales team to make the change some of our systems, the sales ops team specifically did and we had to get on top of their backlog. And that is not an easy thing to do, because they had a bunch of the things they wanted to do first. But this week deemed to core and we ended up proving budget to hire contractors to help them on [inaudible 00:52:38]. It's been a pretty cool to see the results.
Wes:
Awesome. And one of the things I always kind of noticed after I talked to a bunch of teams going through this whole transition from sales at a product-led is I usually notice there's a few quick wins early on to start building the momentum and getting people on board with this. So what were some of those first quick wins that you noticed when you were launching this free plan that gave you a chance to really celebrate with the rest team? Like, "Hey, we're seeing progress here, let's jump on board with this a new way of doing things."
Todd Olson:
Well, they said this ops stuff was like to me, one of the most interesting cool pieces of it, but the product had no self-provisioning built in. So I think the really cool quick rate is that we have now full self-provisioning for all of our product stuff based on what people bought and we have an in-app kind of enforcement. So one of the gates I guess, that we have or triggers for conversion is around how many [MAUS 00:53:43] a customer have. And the product had never had this sense of knowing in terms of what you're supposed to have versus of course, we know what you do have. So now, building all those mechanisms in it now unlocks the ability for us to communicate to all customers independent they're free or not. If they're over what they've bought in terms of MAUS, which guess what drives product-led growth. Just the act of telling the customer you're over, we don't even have to always call them and ask for more money. Sometimes they call us like, "Hey, I noticed that product." It says that we're over like do we do anything for that or how do we pay you more for... It creates opportunities and I know it seems simple, but those little details, not a lot of work behind the scenes happen to make that little thing pop up, but that was one of the cool wins that we did early on.
Wes:
And I know this is kind of controversial or some people that really like disagree with this other teams are all for it like another team or a self-serve team at drift has like a revenue number, but what do you feel about, I give the future of product, do you feel like it's headed in a direction where product teams are going to start owning more of the revenue numbers and having KPIs specific for that, or do you feel it's still going to be a more traditional like it is today?
Todd Olson:
I think it's a really cool idea to have product team own a revenue number and I think absolutely we'll head in that direction. And you're going to hit someone who's super competitive that's going to challenge some sales teams. We're going to drive more revenue than your sales team and then people are going to try to do it. No, I think it's a great idea. So I mean, the question is what is the metric of success? And if you're trying to actually drive revenue with it, then I think that's a great way of thinking of it. So it's funny, I was talking to... I don't want to divulge the company, but one of the cooler stories I've heard recently is a company that everyone's heard of, but I probably shouldn't do it because it was pretty internal to the company, but they have a complete like a great free product and they don't really try to convert it at all.
Todd Olson:
And they get honestly so much revenue from there. There are other channels that they can kind of let this pull even though that the kind of over thresholds a lot of their customers, but every once in a while, when a quarter is tight, they go to this free base and look at the people that are way over in their averages and basically asked them for money. And then that like fills in the gap of their quarter, which is a really fascinating way. They kind of let people steal it for some period of time, not even care, but every once in a while they'll go try to enforce it just to get the numbers up a little bit and it's kind of a different way of thinking and other companies super successful.
Todd Olson:
I don't think all of us could do that, but it's kind of a cool idea. And we're actually, we've been thinking experimenting as well about we can have people, we have actually converted a few people in free. Not trying how, we're just telling people feedback like you're over, we're just telling him they're over. And people don't like being over on things. It's crazy how then they come to you even though we're not shutting them the usage of the product or anything like that and you can still keep using it. Everything's fine by our standards but it's kind of interesting that this just gave people feedback what it does.
Wes:
Yeah. And I've seen that converted to a really good job of that too, when you do have the free products, you can go into new categories and just completely disrupt it like they introduced the landing page software. So there is that's completely free and you can just use it there, I mean, that space itself is super competitive already. There's Instapage unbalanced so many other players and they just come in and say, "It's completely free, use it." And when you get subscribers, then you want to email them like, "We do have our main product over here, but this is completely free for you." So it does give you some really interesting opportunities from even if you think of this free product as just for your marketing team as LeadJen is a very high quality way of LeadJen where you're providing a ton of value to people.
Wes:
So it's rethinking the way you approach marketing instead of writing 1,000 articles on how to do better landing pages and getting that same traffic, you could actually just help them build that with a product. So I think it's super fascinating when we look at the future of marketing as well and how this product-led mindset can truly impact every single team. I truly believe that when we start thinking more product better, we started thinking a little bit smarter and high level, how can we help people really get to the problem and solve it? So I know we're running out of time here, but I want to end on one last question. So you have or in the process right now, going from the sales art to the product-led, what's one thing you would do differently?
Todd Olson:
What was one thing I would do differently? I mean, I've always just want to move faster. So the one thing I would do differently is we ended up in a place where we did end up giving far more features away, but it took us too long to get there. I think people's fear, uncertainty and doubt kind of drove lot of the early decision-making and early versions of the product that I saw in specs, they constrain the product so much that it didn't have any soul. I think all good products have to have some sort of soul. And so now it does, but I think getting to that point it takes us a lot longer than I would have liked just because of the fear.
Wes:
Yeah. And I know we had another speaker from Vendasta, her name's Jacqueline Cook, and she just basically want to say this whole transition from product-led it never ends and it never will ends and you just have to accept that and always be improving. So I think that's a good note to end on here. This stuff never ends. How motivating is that? On that note, thank you so much Todd, for coming on here to really share your thoughts around is how to build this product-led organization. This is a lot of fun and for everyone else who's still here, if you want to join us on the next call just is having right after this. We have someone from Sprout Social who's just going through how they push product at marketing. So if you want to jump just sent to the link there, but for everyone else, thank you for joining us today. This has been a ton of fun, and we'll be in touch soon. Thanks again, Todd.
