Eric Ries’ 2011 book The Lean Startup placed the idea of the lean enterprise in the mainstream. It was Ries who popularised concepts such as the Minimum Viable Product (MVP) and pivoting, which are now essential things everyone in business, from startup owners to business school students, should know.
The lean startup method puts experimentation over upfront planning, iterative design over large-scale development, and measurable customer feedback over intuition.
Lean experimentation is at the core of this approach. In this post, I want to show you:
What lean testing is,
Why it's so important for organizations,
And how to make it work for your business.
I will also share some tips on how to run efficient tests backed by examples of lean experiments to help you start gaining insights and translate them into measurable results in no time.
Lean experimentation is best understood as a part of the Build, Measure, Learn cycle which applies testing and experimentation to products and services, but also entire business models and organisations.
Here are 3 key benefits of lean experimentation. It:
Enables organisations to iterate faster, develop better products and serve customers more efficiently.
Helps impact the bottom line by increasing profits and reducing waste.
Creates a culture where experimentation is the driving force behind the lean startup methodology and an essential step in optimising conversions.
Lean experiments always begin with a hypothesis.
It might seem counterintuitive, but the idea is to conduct these experiments in a way that allows them to fail. In fact, it’s very reasonable and simple: let it fail while you’ve only invested a fistful of dollars in it, rather than when your bank account goes dry. You also need to define how to measure its success.
Lean experimentation brings the best results in cultures that accept occasional failures. That's why the right mindset is the foundation of a successful culture of lean experimentation.
You can conduct an experiment that aims to identify the percentage of participants required to engage positively. If the test fails, you'll know the reason behind that failure and correct your direction on the basis of the new insights.
As you run experiments and validate your assumptions, you will start learning about the real needs of your business.
There are many ways in which one can test a hypothesis.
Here are 3 essential pointers to help you create smooth and straightforward tests:
Make your tests small and systematic – don't put your organization in the position where you're testing your entire venture or too many variables at the same time. That way you risk becoming overwhelmed and eventually failing to implement what you've learned. It's best to make your tests small and regular. Choose a single problem, area, or variable. Create a hypothesis and test it systematically.
Find the right tools for testing – many tools which come in handy for testing are free or cheap, opening the doors to more elaborate solutions when you get enough valuable insights from your initial tests and can justify that investment.
Analyze, track, and measure – before testing, you need to have a technical framework in place that will help you track and measure your success. That's how you'll be able to see your progress and analyze your results. It's also easier to share and implement your findings when you have all the information in one place. Equipping your team with the right tools is one step – investing in the right configuration early on is another one.
It may look complex but in fact it’s simple – and I mean it. The hardest thing in it is having the proper mindset. The rest can be done by the development partner you have, provided of course that they share the same understanding of it.
The Lean Experiment Canvas (LEC)
The cornerstone of the lean startup methodology is the Lean Experiment Canvas (LEC) that allows you to run smarter experiments with better results that are in line with the Build, Measure, Learn cycle.
It's risky to leave learning and insights to chance. The LEC model helps solidify assumptions, risks, and metrics before you start testing. The objective of LEC is supporting businesses that want to uncover truths about themselves and their customers.
The model is made up of 12 blocks that collectively take organisations through the scientific method of lean experimentation. All blocks are equally important – but as you fill out information, you're bound to discover a unique pathway for your organisation to follow.
See also: 10 UX tips to improve website navigation
Now that you know the basics, let's have a look at the different types of lean experiments:
Wizard of Oz tests
This type of testing is conducted by a human who manually performs specific functionalities of a product or service. For example, if you'd like to find out whether investing in a chatbot is a good idea for your business, you can open a chat service on your website that is managed by a human who poses as a bot. You will learn what kind of questions people ask and how they interact with the chatbot.
Another smart way of testing is interviewing people and asking them open-ended questions to learn about their explicit and implicit needs. As you pose these questions and get answers, you will start to notice patterns that indicate what users find challenging and why. This type of insight will direct you toward developing a solution that makes sense for users.
Another interesting type of testing is based on creating a prototype of your solution and putting it in the hands of users to see how they respond to it. You will immediately get an influx of thoughts or concerns people might have about it even before the product is publicly available.
Smoke testing is based on testing a service that doesn't exist yet to understand whether there is interest and engagement for it. A crowdfunding campaign is an excellent smoke test. Or you can just put a sign-up form on your landing page and see how many people subscribe to learn more about your offer.
For more information, check out this comprehensive overview of lean testing methods.
Do you have any questions about lean experimentation or other UX design services? Give us a shout out in the comments; we're always happy to start a conversation that enriches the community and helps us all deliver better products and services.