Product metrics for Engineering Managers

[00:00] Welcome and Introduction to the Livestream

Andrew Murphy: Hello. Good morning, everybody, if you're in a place where it's morning. Good afternoon if you're in a place where it's afternoon, and good evening if you're in a place where it's evening. And welcome to another Tech Leaders Launchpad live stream. If you don't know me, I'm Andrew Murphy. I'm the founder of Tech Leaders Launchpad. I help engineering leaders understand the skills and mindsets they need to succeed. When I became a tech leader 15 years ago, I went home Friday a senior engineer, and came into work on Monday a tech lead. Nobody taught me the difference. I had to work everything out the hard way, make mistakes, and there weren't many resources back then. Now, I try to make sure you don’t have to do things the hardest way. One way is through these livestreams, where I share the people I learn from, people I know in the community who have great ideas, who can talk about topics important to tech leaders. And one of those people is Erwin. Welcome, Erwin, to the Tech Leaders Launchpad livestream.

Erwin van der Koogh: Super, super excited to be here, Andrew.

Andrew Murphy: Do you want to give the peeps at home a little background about yourself—where you've come from, where you work now—and segue into this topic? When I ask guests what's important to them, what they think tech leaders should think more about, this is what we landed on. So, give folks a sense of your background and why this topic's important to you.

[01:51] Erwin’s Career Story and Discovering the Importance of Product Metrics

Erwin van der Koogh: Yeah, I completely relate to your story—leaving work Friday as an engineer and returning Monday as a manager with no one explaining how those roles are different. They’re very different, who would’ve guessed? And the skills that make you a great engineer don’t necessarily make you a great engineering leader or tech lead. I’ve had a varied career, always in tech, but the recurring theme for me has been “how do I have more impact?”

When I started as a developer, the way to have more impact was to become a better developer. But over time, I realized there’s diminishing returns there, so teaching other developers to become better made more impact than just me improving. So I moved into architecture, then realized organization structure was a bigger problem than code. That got me involved with agile in the early 2000s. Eventually, still in pursuit of having greater impact, I moved to Australia, started a startup, and that’s when a major insight hit me: understanding how a company makes money is one of the most powerful skills. If you want to help or improve a company, you have to know how it makes money.

But companies actually exist to make a change—they need money to make that change happen. So, understanding how the business functions, how it gets customers, how it gets more money per customer—those are key. That’s why this topic of product metrics is so important to me: It’s essential for engineering leaders to understand the business if they want to have real impact.

[07:09] Fractal Impact, Business Value, and Moving Beyond Tech Metrics

Andrew Murphy: It’s almost a cascade or even fractal: you ask “how does the business make money?” and the surface answer is “customers pay for software,” but then you have to go deeper. Why do customers pay for the software? What value does it add for them? How does it fit into their lives or jobs? It’s a fractal understanding of value—drilling down into every action’s impact.

This segues into today’s focus: as engineers, we’re often obsessed with technology metrics—things like DORA and SPACE, measuring system functioning. You work at Honeycomb, and observability is a big part of that platform—seeing technology metrics, performance, errors. But that’s not what we’re talking about today. Could you dig into what we mean by product metrics, and why are they important?

[08:42] Defining Product Metrics and Tying Them to Business Value

Erwin van der Koogh: When I talk about product metrics, I really mean your business metrics—that’s what they’re tied to. Technical metrics are important, but they’re supportive. Like you said: how do we add value to our customers? Why are they paying us?

Unless you’re running an unethical or illegal business, you can only capture a percentage of the value customers perceive. You can’t charge me $100 if I only feel I’m getting $20 of value. That’s why it’s so important to figure out how to deliver more value, because you can only ever capture a fraction of it. So, your levers are delivering more value, getting more customers, and getting more money per customer. There are three ways to deliver value:

1. Creating more revenue
2. Protecting existing revenue (important in mature markets—protecting from loss, fines, competitors)
3. Saving cost (ideally by reducing unit cost across-the-board, not superficial cost-cutting like limiting bananas in the office)

[13:06] Leading vs. Lagging Indicators in Product and Tech Metrics

Andrew Murphy: That leads to the concept of leading and lagging indicators. Sure, revenue per customer is the ultimate thing we care about, but it’s a lagging indicator—something we only know well after actions are taken. In the early days, or with a new product, it may take 6, 12, even 24 months before you see that in results. Before then, you have to value leading indicators: site visits, trial signups, people talking about you.

In tech, we have analogous metrics: for instance, DORA’s lead time for changes or change failure rate are lagging indicators. But PR size or complexity can be leading indicators: not valuable on their own, but correlated with the outcomes we care about. Fundamentally, these concepts apply whether we’re talking product or system health.

[15:56] Understanding Leading and Lagging Indicators for Freemium & Product Funnels

Erwin van der Koogh: That’s a great point; leading indicators are things you trust to bring value in the future, even though they don’t result in “money in the bank” today. For example, with a freemium product, your free users cost you money to serve, but the bet is that a certain proportion will convert to paid. So total free users is a classic leading indicator; it’s only valuable if you know the conversion dynamics.

[17:56] Engaging with Livestream Viewers & Multilingual Questions

Andrew Murphy: Yes, by the way, Arti is saying good morning in Japanese (I had to use Google Translate!). That’s a reminder: if anyone has questions for Erwin or me about product metrics, Honeycomb, or Tech Leaders Launchpad, please post them. The benefit of a livestream is we can respond to your questions directly. If it’s in another language, be patient as I do more Google Translating.

Erwin van der Koogh: I’ll happily respond to Dutch questions—though it’s probably the wrong timezone for those folks!

[19:19] Choosing Valuable Metrics: Avoiding Vanity Metrics

Andrew Murphy: Next, how do you choose which leading indicators matter? How do you rationalize which metrics to care about when their connection to revenue may be indirect or unclear? And how do you avoid vanity metrics?

Erwin van der Koogh: Vanity metrics are numbers that look good because they go “up and to the right,” but don’t actually represent value creation. Free user signups can be vanity metrics if they don’t correlate with conversions or value delivered. Context matters: metrics can be “vanity” depending on your business model. If you can’t demonstrate a relationship between, say, page views and subscriptions, then tracking page views is pointless.

[22:15] Concrete Example: Vanity Metrics in Online Courses

Andrew Murphy: Concrete example time! When I first started making training courses, I uploaded them to Udemy. I got thousands of enrollments and thought that was amazing. But when I looked at active learners and “minutes taught per learner,” I saw that only about 0.1% of enrollees actually started a lesson, and only a tenth of those completed more than a handful. That’s why I created Tech Leaders Launchpad—to help a smaller, more engaged group, rather than chasing big vanity numbers.

[23:59] The Value of a “One Metric That Matters”

Erwin van der Koogh: That’s a perfect segue to the idea of a single, central metric for your company: the “One Metric That Matters.” For you, it’s “lives affected”—how many people you’ve helped transition from individual contributor to manager or leader. At Honeycomb, our core metric is reducing “mean time to WTF”—that moment of confusion when looking at data. For us, success is customer enlightenment and learning from their data.

[27:34] When Vanity Metrics Offer Value—and How Funnels Inform Us

Andrew Murphy: Vanity metrics aren’t totally useless—you just have to be honest about their meaning. For example, Tech Leaders Launchpad’s core metric is lives changed, but engagements, sign-ups, and page views are still meaningful as funnel diagnostics. Changes in those numbers can indicate problems or improvements in specific stages (for instance, split-testing two sales pages).

[29:25] Vanity Metrics Become Leading Indicators—If Linked to Impact

Erwin van der Koogh: As long as you know how a metric ties to your core outcome, it’s not truly a vanity metric—it becomes a useful leading indicator. For Udemy, maybe the funnel is inefficient, but so long as enrollment numbers ultimately connect (even distantly) to outcomes, tracking them is useful. It’s about line of sight.

[31:16] Organizing Metrics Around Value Delivered, Not Just Money

Andrew Murphy: There’s something implicit in our “one metric” examples—they’re not simply revenue-based! Both Honeycomb’s and TLL’s north stars are about user value, not just money. This helps mitigate burnout and misaligned incentives: if you continually focus on revenue, things can feel demoralizing. Anchoring your work around improving lives or customer outcomes is more motivating and productive.

[34:02] Delivering Customer Value (and Brief Pricing Sidebar)

Erwin van der Koogh: That brings us back to the earlier point: you can only capture a percentage of the value you deliver, so focus relentlessly on maximizing delivered value. Pricing is complex and another topic entirely; as a founder, pricing may become your headache, but for most engineering leaders, focus on value delivered.

[35:25] The Engineering/Customer Success Info Gap

Andrew Murphy: In many organizations, there’s a huge information gap between engineering and departments like customer success or support. Those teams field lots of workarounds and pain points that engineers never hear about. I’ve sat with support and watched them help users “work around” bugs no one on the dev team even knew existed. Have you seen the same, Erwin, and any tips for bridging that gap?

Erwin van der Koogh: Absolutely. At a bank, I once took engineers down to the branch office to observe people using our software. We saw someone redacting multiple fields in a PDF, and every time, the app would close and they’d have to reopen it, repeating the process. Turns out, it used to only ever require one redaction, but business rules had changed. It was a trivial fix, but no one ever told engineering because “that’s just how it works.” The lesson is: as an engineering manager, your job isn’t just tech—it’s connecting with other departments to understand how your work impacts the business as a whole.

[39:14] The Importance of Talking to Users and Cross-Team Communication

Andrew Murphy: Advice for any new leader joining a company: spend your first few months talking to as many users as possible. Metrics are great for scaling your learning, but nothing replaces direct, human-to-human context.

Erwin van der Koogh: Consulting is even more extreme—you have to get those conversations going fast. My trick: join the “smoke break crowd” (even though you don’t smoke)—it’s a cross-section of the company and gets you a pulse on what’s really going on.

[41:21] How to Get Good at User Empathy and Building Connections

Andrew Murphy: It can feel harder as a consultant, since you’re always on the move, but that’s only more reason to practice and get better.

Erwin van der Koogh: Early in my career, I asked a colleague how he built such great customer relationships. His answer: “Just drink coffee with people.” I thought that was useless at the time, but it’s really about genuine, active listening and showing you want to understand.

[43:23] Framework: The Pirate Metrics (AARRR) for Product Health

Erwin van der Koogh: Before we wrap, I want to mention my favorite framework for product metrics: Pirate Metrics, or AARRR. It’s not a list of five metrics, but five groups or areas to categorize your most important metrics:

• Acquisition: How do we get new users or customers in the door?
• Activation: How do we know new users are truly engaged—what’s their “aha” moment?
• Revenue: How are you actually making money?
• Retention: How many users keep coming back or renewing?
• Referral: Are your customers spreading the word and bringing in others?

Andrew Murphy: Exactly. These are labels—you define what each means for your business and what indicators matter for each.

[49:43] Further Notes on Revenue, Retention, and The Ethics of Monetization

Andrew Murphy: Net Dollar Retention is an important metric for SaaS—if you’re upselling or expanding use, you want that number over 100%. It’s often easier to grow revenue from an existing customer than to win a new one.

Erwin van der Koogh: Yes! And to the point about money: there’s nothing wrong with charging for the value you deliver, as long as it’s honest and your customers actually benefit. I’ve seen situations (like telcos not notifying users about price drops) that are clearly unethical; don’t do that.

[54:09] Learning More: Starting Points and Resources for Product Metrics

Andrew Murphy: If you’re a manager wanting to learn about product metrics—CAC, LTV, etc—start with Pirate Metrics for the big picture. We’ll share links as well.

Erwin van der Koogh: And if you want to learn about all the acronyms (CACs, COGS, OPEX, CAPEX, etc), the book “Personal MBA” is a great resource.

[55:50] Wrapping Up: Tech Leaders Launchpad News and Next Live Streams

Andrew Murphy: To close, if you liked this stream, check out the TLL newsletter (QR code shown). You’ll get leadership resources, articles, updates on future livestreams and events. We’re also running in-person workshops across Australia and New Zealand on leadership transition and conflict resolution—details via the QR code. Next stream: Anita joins to revisit the topic of neurodiversity in tech teams—an area that’s seen a lot change in the past few years, some good, some not so.

Erwin van der Koogh: As a person with ADHD, happy to share: you’re definitely not alone in tech! And it’s not just ADHD—there’s a range of neurodiversity.

Andrew Murphy: Likewise—diagnosed on the autistic spectrum, and I’m not the only one. Erwin, where should people find you if they want to learn more?

Erwin van der Koogh: Connect with me on LinkedIn. Happy to chat!

Andrew Murphy: Thanks to everyone joining, and thank you again, Erwin. See you next time on Tech Leaders Launchpad!

Erwin van der Koogh: See ya.