As a business leader looking to develop a new product or enhance your current product’s functionality, there are myriad trade-offs to evaluate and decisions to make. The stakes are high, especially when your product timeline is directly tied to the velocity of decision-making.
Without a common framework to ground your decision-making, decision fatigue can set in, leading to project delays, low team morale, or even worse: a product that fails to launch. Successful product development is built on a solid foundation of experimentation and evidence.
Build-Measure-Learn
The “Build-Measure-Learn” framework, as introduced in The Lean Startup, is at its core a feedback loop. The dev team builds a Minimum Viable Product (MVP), tests the product with stakeholders, and learns from their input. The team then incorporates what they learned into a new phase of the build.
“Build-Measure-Learn” is a great methodology to use when you need to get a dev team moving quickly, but it isn’t without planning efforts. In fact, pretty significant planning is required to pinpoint the business objectives and to make a reasonable hypothesis about the results your actions will yield.
“Plan-Hypothesis-Build-Measure-Learn” isn’t quite as catchy as “Build-Measure-Learn,” but without those key steps, “Build-Measure-Learn” cycle can quickly deteriorate to “Build-Measure,” or just “Build.”
Evidence-Based Product Development
Building astonishing products requires evidence-based product development. Evidence-based product development starts with a dev team that understands the business objectives -- the “why” -- at a deep level and allows that “why” to inform each decision they make. A dev team who keeps the “why” front of mind continually evaluates how their decisions will impact not just the product they are building, but the overall business objectives.
Once the why is established, it’s time to formulate a hypothesis -- an educated guess about what will happen as a result of a specific intervention. Here’s a very basic example: let’s say your company’s “why” is reducing the friction users experience. If your desired outcome is to increase conversion, one hypothesis might be “If we do not ask for a phone number in order to sign up for gated content, we will see an increase in conversion.” This is a simple use case, but you can apply the same process to tackling larger decisions.
Next, it’s time to experiment vis-a-vis your build and rounds of feedback gathering. As your team shares what they’ve built with stakeholders, the project lead will want to gather feedback and measure results. Remember, you are looking for evidence to indicate if your hypothesis was true or false. Using the results, you can formulate a new hypothesis and adjust your build accordingly.
Over time, the evidence you’ve gathered becomes information you can use -- and not just for this build, but for future projects, too. This becomes information you can rely on, even if the initial rationale for a hypothesis was a bit unconventional. For example, let’s say your CEO has a hunch that a certain design decision will increase engagement: it’s okay to include that in your hypothesis. Name and document the hypothesis, run your experiment, and measure the results. Over time, you’ll have data to back up the CEO’s hunches… or not!
Now comes the hardest part: applying what you learned. When you have a defined hypothesis and results to measure it against, there’s no room for confirmation bias or just explaining away results and proceeding as planned. Truly learning means following the facts where they lead. Decisions can be grounded in facts, based on the results of your experiment.
Was your hypothesis correct? What does the data indicate that your team should do differently in the next sprint?
If you don’t form a hypothesis, run a scientific experiment, evaluate the evidence, and let the evidence inform your next steps, you are losing valuable learning. As a business leader, you want to learn not only what’s going to be successful for your product, but what your team is capable of and where there are gaps.
What about Small Decisions?
While evidence-based product development is a simple framework for decision-making, you will undoubtedly have times when you need to make a decision quickly due to constraints on time, funding, staff resources -- or all of the above. In that case, acknowledge that you can’t efficiently form a hypothesis, experiment, and measure on each and every decision. Focus on the biggest questions and work your way through in order of priority. Name your constraints and identify the business outcomes. Based on past data, which decision is most likely to lead to a result that’s nearest to your desired business outcome?
Pro tip: If you are having a difficult time choosing between options, consider quantifying your variables according to a model like the one shared in Smart Choices, where each variable is given a numerical value and a weight. The choice with the highest points wins.
Tracking the Outcomes
Your stakeholder feedback and measurements will provide you with valuable data. Don’t lose track of it! Some leaders will choose a tool built for tracking goals and progress made towards OKRs. However, if you’re just getting started with evidence-based product development, resist the urge to over engineer. If a collaborative tool like Google Sheets, AirTable, or a wiki works to get your team started, begin there.
Once you get into the practice of documenting them, incorporate your hypotheses and measurements into your project documentation, just like you would other artifacts. Review a bullet-point version of your hypotheses and measurements at your sprint retros or post mortems for a comprehensive summary of what you’ve learned through the course of the project.
Evidence-based product development provides us with some needed guardrails, while still allowing the flexibility to pivot as needed when contingencies come up or priorities change (as we all know they do). It provides us with a consistent mechanism for evaluating how our assumptions impact our work. What’s more, it gives us the confidence that our teams are aligned to the business’s objectives.
Want to learn more about how Presence builds astonishing products? Send us an email at contact@presencepg.com to begin the conversation!