Growth Experimentation Velocity
Build a high-output engine to compound small wins into massive growth.
Help the user with growth experimentation velocity using insights from 10 guests and posts across Lenny's Podcast and Newsletter.
How to Help
- Establish the Baseline - Analyze current conversion funnels and identify the single North Star metric to focus on.
- Prioritize and Plan - Use frameworks like ICE or RICE to rank experiments by impact and engineering cost.
- Execute and Iterate - Launch scrappy tests quickly to find signals of life before scaling into robust features.
- Scale and Socialize - Systematize the sharing of wins and failures across the organization to multiply the impact of every insight.
Core Principles
Search for signs of life
Timothy Davis: "You can always do a very, very small test. You can just put a little money into a platform, see if there's a sign of life. If there is, then you can pull back and say, 'Okay, we have signs of life. Now let's build a campaign around that.'"
Validate new channels or ideas using low-budget tests and narrow match thresholds before committing significant resources.
Embrace the counterfactual
From "How today’s top consumer brands measure marketing’s impact": "Testing/conversion lift studies (CLS): regularly run by marketers to validate what performance would look like if you switched a channel off, or scaled spend up or down."
Use randomized testing and lift studies as the gold standard to observe what would happen without your intervention.
Leverage compounding effects
From "The secret to Duolingo’s exponential growth": "To get the best long-term gains, you should always have a sense of urgency. The quicker you launch winning experiments, the quicker those changes impact your growth. Not only that, but these improvements compound!"
Focus on high experiment velocity because early small wins multiply over time into significant competitive advantages.
Optimize psychological commitment
Jackson Shuttleworth: "We've actually set up really good infrastructure for copy testing. We used to say continue, our standard CTA is continue, and we changed that to commit to my goal, and it was a massive win."
Shift from generic microcopy to intentional language that reinforces the user's specific goals and psychological state.
Lower friction with scrappy tools
From "Fostering a culture of experimentation": "When systems are still in flux, you don't want to overinvest in tooling that will become outdated immediately when your data schema gets updated or some other piece of infrastructure changes. However, it is essential to have a way to rapidly iterate, and that means quick access to experiment results data. So if you need to in the early days, build something simple and scrappy at first, and over time evolve it to support the team's needs."
Prioritize rapid iteration over perfect infrastructure by starting with simple internal tools to prove the value of testing.
Templates & Frameworks
- EVELYN (Experiment Velocity Engine Lifting Your Numbers) - Airtable Template (Introducing DRICE: a modern prioritization framework) - A batteries-included Airtable template for managing growth experiment prioritization using RICE/DRICE
- Noom's Experimentation Velocity Principles (How to win in consumer subscription) - A set of operating principles for running a high-velocity experimentation program in growth
- 4-Step Conversion Optimization Process (Prioritizing conversion opportunities) - A structured end-to-end process for identifying, prioritizing, executing, and learning from conversion optimization work
- Experiment Design Template (Breaking into growth) - A Google Doc template for designing and running growth experiments
- Impact and Learnings Review Meeting (Ben Williams) - A weekly document and meeting structure used by growth teams to discuss and socialize experiment learnings.
- 6 Guidelines for Experiment Urgency (The secret to Duolingo’s exponential growth) - Tactical guidelines for moving quickly on experiments to maximize compound growth, used at Duolingo
- Growth Ideas Brainstorming Framework ('How Might We…?') (Growth ideas) - A facilitation approach for running team brainstorming sessions where you go through a categorized list of growth ideas and apply 'How might we…?' framing to ge
- Conversion Optimization Decision Tree: Experiment vs. Ship (Strategy and tactics for increasing conversion) - Guidance on when to A/B test conversion changes vs. when to just ship them
See references/artifacts.md for the full list with details.
Questions to Help Users
- "What is the single North Star metric you are currently trying to move?"
- "How many experiments are you currently running per week?"
- "What is the estimated engineering cost versus the predicted impact for your top three ideas?"
- "Do you have a standardized process for sharing experiment learnings across the whole team?"
- "Is your team autonomous enough to launch experiments without multi-level approvals?"
- "What percentage of your user base actually encounters the flow you are planning to optimize?"
Common Mistakes to Flag
- Waiting for silver bullets - Teams often stall growth by looking for one massive feature instead of accumulating many small optimizations.
- Paralysis by testing - Applying rigorous A/B testing to every minor change can slow down execution if there is not enough data volume.
- Ignoring the addressable pie - Failing to factor in how many users actually see a change leads to overestimating the real-world impact.
- High-friction approvals - Requiring multiple levels of sign-off for experiments kills the momentum needed for a high-velocity culture.
Deep Dive
For all 16 sourced insights from 10 guests, see references/guest-insights.md
Related Skills
- Growth Model
- Acquisition Channels
- User Onboarding Activation
- Retention Engagement
