Different Approaches to Product Development: How Buzzvil’s Product Team Works, Part 3

What began as a series in late 2021 has somehow reached the end of 2022. Over this year, the way Buzzvil's product team works has changed even more. Change has outpaced my writing, making it difficult to complete the six installments I originally planned. So in this third and final post, I will briefly bring the broader picture together.

In this series

1. A customer-centric strategy: reorganizing groups and using NPS
2. Product vision, strategy, and roadmap: the product strategy pyramid
3. Different approaches to product development

How we work

Product Requirements Documents (PRDs) and OKRs

Recently, a tweet claimed that OKRs were a Google conspiracy to slow down its competitors. Google's CEO Sundar Pichai replied, “Oops. Finally someone figured it out:)” and received plenty of likes. He was joking… right? OKRs can help an entire organization focus on achieving its goals, but when used badly, they can also slow it down. Hence the conspiracy theory.

Oops. Finally someone figured it out:) https://t.co/gh36taeElF

— Sundar Pichai (@sundarpichai) July 2, 2022

We need to be careful about what “slow” means here. Spending too much time administering OKRs can certainly slow execution. But poorly run OKRs can also slow learning and changes of direction, bringing the growth engine to a halt. The first problem is easy to notice and address quickly. The second is far more dangerous because it may take a long time to recognize.

One common mistake is defining key results as nothing more than a to-do list. If you list the quarter's tasks and measure KRs only by whether those tasks are complete, it is hard to tell how the work affects the ultimate objective. Suppose your objective is to improve user retention, your plan is to build five features, and your KRs track their completion. You quickly build and launch the first feature. User feedback then reveals that the main cause of poor retention lies elsewhere, and none of the five planned features will solve it. What should you do? If you become fixated on achieving the OKRs and stick to the original plan, you could end up with an excellent OKR completion rate while retention falls.

We need to separate KRs from initiatives—the plans for achieving them. As I emphasized in the previous post, vision and strategy come first, followed by roadmaps and plans. Once you define the OKRs, express how you intend to pursue them as initiatives. Those initiatives should change quickly as circumstances change and you learn more.

An example of OKRs and initiatives:

Objective	Key Results	Initiatives
Improve user onboarding and activation.	Increase the profile completion rate from 60% to 85%.	• Show completed profiles on more screens. (PRD) • Split profile fields into steps and display them progressively. (PRD)

A Product Requirements Document (PRD) works particularly well with this approach. It captures the requirements for a particular product or feature and defines what it needs to do and why. A PRD includes the purpose, problem to solve, requirements, use cases, and constraints. Be careful, however, not to prescribe implementation details, UX, or design too specifically. Leave room for the team to find better approaches.

The combination of OKRs and PRDs is something Buzzvil's product team continues to work at, even through substantial change.

The 4 Disciplines of Execution

The most talked-about book at Buzzvil in the third quarter was probably The 4 Disciplines of Execution, which we called “the red book.” Its Korean title is rather long and awkward, so the cover color seems to have become its nickname. It helped us address several areas where we had struggled with OKRs.

The authors propose four disciplines: 1) focus on the wildly important, 2) act on lead measures, 3) keep a compelling scoreboard, and 4) create a cadence of accountability. They sound simple, but putting them into practice requires more than changing a system.

Buzzvil had also been struggling with too many goals and lagging indicators beyond our control. Some challenges remain, but choosing this book for our leadership sessions helped us improve a great deal. Acknowledging the “whirlwind”—work that must be done but does not directly serve the most important goal—helped us narrow our goals and pursue them more forcefully. We began looking for controllable lead measures when setting KRs. Scoreboards made the status of our most important goals visible at a glance, to anyone, at any time.

Here are a few examples from the product team:

• Define the most important goal. Before setting OKRs, we discuss what matters most, such as a financial or operational goal. We also mark the most important goal within each team's OKRs.
• Show targets on the scoreboard, or dashboard. A dashboard that shows the target for each point in time helps a team focus more effectively than one that only reports the current state. Showing whether the team is winning or losing against its target prompts action.
• Set goals that reduce the whirlwind. We acknowledged the whirlwind and set goals to address its causes upstream.
• Find controllable lead measures. Amazon's Working Backwards also emphasizes their importance. If we set a target using a lagging indicator such as revenue, the distance between our actions and their results makes it hard to learn and improve quickly. Defining actionable lead measures that are more than a to-do list remains difficult, but Buzzvil's product team works to set better goals each quarter.

Opportunity Solution Trees: focus on opportunities before solutions

Ideas are cheap. Execution is everything. — Peter Drucker

Becoming attached to ideas seems to be human nature. A good idea feels exciting and satisfying. But becoming too attached can trap us inside the idea instead of helping us create value. A good idea that fails to solve a real customer problem belongs in the graveyard. We should send more ideas there than we expect, yet too many make it into a product first and take the product to the graveyard with them. That is what happens when we focus on solutions rather than problems.

An Opportunity Solution Tree (OST) helps us focus on opportunities before solutions. First, choose a desired outcome. Then list the opportunities that could help achieve it, and attach potential solutions beneath those opportunities. Prioritization happens at the opportunity level, rather than at the solution level.

One of the most important things to remember is that opportunities must come from customers. If you do not define them through insights from interviews or data, you can end up inventing opportunities to justify solutions you already wanted to build. This is not easy, so we take particular care when using OSTs at Buzzvil.

Pre-mortems: identify the riskiest assumption

Eric Ries, author of The Lean Startup, emphasizes the importance of the riskiest assumption. He argues that when building a business, you should center your validation and execution around it. Even if everything else goes well, getting that one assumption wrong could prevent the whole thing from working.

A pre-mortem helps identify the riskiest assumption by simulating the future before an initiative starts. Discuss why the initiative might have succeeded, or why it might have failed, to find where you should focus.

Team Topologies: reduce a team's cognitive load

Complexity inevitably grows as services and organizations expand. If you focus only on short-term development speed, you may eventually find that the whole organization has slowed down. Without deliberate effort, dependencies between teams increase, leaving more and more things to consider just to plan and release a single feature.

Team Topologies proposes ways to organize teams so they can reduce cognitive load and move quickly and independently. It defines four team types—Stream-aligned Teams (SATs), Complicated Subsystem Teams, Enabling Teams, and Platform Teams—and ways for them to interact.

We are working to apply Team Topologies to Buzzvil's circumstances. Through event-storming workshops with multiple stakeholders, we defined bounded contexts. Following the Inverse Conway Maneuver, we are pursuing a team structure that supports the product architecture we want. There is no silver bullet that works everywhere, of course, so we are simplifying and adapting the approach to our current organization. Above all, it is important that the people discussing team structure share an understanding of the concepts behind Team Topologies.

PPP: Progress, Problems, and Plan

Sharing work updates means more than reporting status. When information flows well, team goals stay aligned with company goals, people stay focused, and the organization can respond quickly to change. Alignment matters enormously, but it can also be demanding and difficult. In particular, updates can become mere formalities if they do not distill what matters to each audience. Too much information can be as unhelpful as none at all.

Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. — Antoine de Saint-Exupéry

Buzzvil's product team uses PPP for weekly updates. We interpret our OKR metrics, share Progress, explain the Problems we have encountered, and lay out the Plan for the next week. These may seem like obvious categories, yet they are easy to overlook without deliberate attention. PPP gives us a reason to think through each one.

North Star Metric (NSM)

KPIs, OKRs, and OMTM are among the many approaches that connect goals with measures. A North Star Metric is a single measure intended specifically to track whether a product and business are growing well over the long term. It should connect to revenue without being revenue itself. Revenue is what customers pay to obtain value. Making it the North Star can encourage short-term growth that damages the business in the long run.

Buzzvil divided its organization into Demand and Supply groups around their customers. Since their primary customers differ, their North Star Metrics should differ too. In late 2021, led by DA team lead Ed, the Supply group defined its North Star as the weekly number of users who met certain criteria, naming it WVU. We found that as more users met those criteria, the value delivered to our publisher customers—primarily advertising revenue—increased noticeably. The Demand group is still identifying its North Star. From the product team's perspective, we are evaluating reach—the number of people who saw an ad—that meets certain criteria.

Wrapping up

Beyond these approaches, Buzzvil's product team uses impact mapping, user journey maps, lean customer development, pretotyping, event storming, Q12 engagement surveys, and other methods to deliver more value to customers. We also try to remember that methodologies are only tools and keep our attention on the principles and purposes behind them.

A great deal has changed in the year since I began this series. Team structures and operating practices have evolved. Growth inevitably brings growing pains, and each time they appear, we need to accelerate learning and act flexibly to keep growing. Our product team has learned and grown tremendously through those changes over the past year.

Our way of working is certainly not the only right answer. But the journey of building a growth engine is surely a step toward one. Thank you to everyone at Buzzvil working to build better products and deliver more value to customers. There are no free lunches and no silver bullets. We will keep learning and improving quickly. We welcome your encouragement, support, and applications. 🙂

Apply for an engineering role at Buzzvil

Have a coffee chat with a Buzzvil tech recruiter