How Do You Measure Success in Design?

Nowadays, many businesses are putting responsibility on design leaders to prove the value of design: The State of Design 2021 study shows that 57% of respondents must measure and report team outcomes to their organization. It is expected from design teams to tie up design with company strategy, goals, and revenue targets. However, it is not easy to measure the impact design has on business. The questions arise: What metrics should be monitored to measure the impact on the business’ bottom line? What KPIs should be tracked and what frameworks to use to measure design success? We invited Senior Product Designers, Heads of UX and Design, and Design Team Leads to share with us their insights into how they measure and report the outcomes of design for their companies.

Defining the right metrics

Different businesses apply different measures and frameworks to evaluate the value and effectiveness of design because no framework works for all organizations. Some companies look into user satisfaction, others look at product response before and after a redesign. However, some experts explain that they do not distinguish between design KPIs and business KPIs and measure what is important - sometimes it is experience, sometimes it is revenue. It is all about the context and business goals.

Darya Luganskaya, Senior Product Designer at Revolut

In design-driven companies, it is not that hard to argue about the importance of design, while it is not always the case for others. To evaluate the effect of design, businesses can set some metrics indicating if their customers are happy with the service, such as the Net Promoter Score (NPS). It shows what percentage of customers would recommend a company to friends and family.

Another useful metric is retention - it shows how many people come back for a second purchase. Churn rate would help measure how many customers stopped using a service over some time and engagement metrics will help understand how many people make more than one purchase. Other factors outside of design can contribute to metrics, e.g. how support operates, so it is important to dig deeper and make sure design changes are tracked well.

Across the company we measure the Net Promoter Score (NPS). It shows what percentage of customers who would recommend a company to friends and family. People are asked to rate on a scale of 1 to 10 how likely they are to recommend this service to a friend. There are specific metrics for each feature developed by a particular team, but I don't think I can share those.

Robin Lantz & Marcus Johansson, Senior Design Leads at Klarna

There’s no easy way of measuring design’s impact on business. Especially because the role of design within a business might be different from company to company. Apple and Maersk probably have two very different setups for designers but both probably have invested in design. We do believe though that design is an integral part of impacting the business performance and that if the design is good you should be able to see how it is positively impacting the business metrics.

Sometimes it can be dangerous to draw the line too hard between business and design. If the business is considered successful and design is a vital part of the organisation - then it might be enough to draw conclusions. We collaborate with stakeholders from engineering, product management, and design to figure out what is most important to measure.

Priya Sinha, Head of Product Development and Customer Solution at Vodafone Group

Business growth is not a 1:1 relationship with product design. Firstly, we need to define the business value the product is trying to fulfil and measure it as a Customer Performance Index (business costs, efficiency or CSI). Next is the need to define top product capabilities with metrics as minimum success criteria. Design performance is measured against these criteria. Some examples could be stock availability, pluggability score etc. We then define clear use cases and draw a boundary on what the product can and can’t do. We ensure all happy and unhappy paths have been considered. Some performance metrics are:

• Metrics on agreed success criteria. Product Usability Index for ease of use of platform and Product Enrichment Index to release features using future-proof designs.
• Easy referrals from customers and constant feedback from real users.
• SLAs and % Fallouts against industry standards. Customer escalation to record empathy.
• Internal KPIs (reduced opex, increased automation) are often missed!

Here are some examples of metrics for design performance that we track:

• Stock availability,
• Inventory costs,
• Product Usability Index, such as the ease of use of a platform,
• Product Enrichment Index to deliver enhanced features within shorter cycles, SLAs,
• Business and technical fallouts and Internal Satisfaction Index (ISI) regarding delivering internal KPIs such as reduced opex etc.

Dave Bowker, Head of Design at GFT

If your routes are being run then you measure satisfaction, if they're not being run then you need new routes! We measure workflow efficiency. When designing software for business, most tasks are either investigative and/or repetitive, so creating an experience that optimizes user workflows is key.

Fernanda Enrigue, Design Team Lead at Lengoo

I will say, however, that using part of the Google HEART framework always makes sense:

• Happiness,
• Engagement,
• Adoption,
• Retention,
• Task success.

Happiness can be tracked with frequent surveys. Adoption and Retention are easy metrics to assess but hard to achieve. User Testing is perfect to measure Engagement and Task Success. Testing on its own can however be difficult to implement, especially in smaller companies under budget constraints and time pressure. But I see it as the only way to really measure the impact of design.

I also believe that showing the value of user testing is our job as designers. There are ways to do user testing without a big budget or long data processing times. It's a matter of creating smart processes and having a strong relationship with your users. We track project times and user experience with the Google HEART framework that I mentioned before. Both of them are aligned with our Product and Marketing KPIs.

Samantha Lim, Senior Product Designer at Finleap

The metrics that companies should track to measure design’s impact should be aligned with the objectives of the redesign. For instance, if the redesign aims to improve conversion rate by simplifying the application funnel, the metric to focus on should be the conversion rate. On top of that, companies can also understand customer satisfaction by

• tracking customer reviews,
• customer feedback,
• sign up rates,
• drop off rates,
• time spent interacting with the product,
• product referrals.

Depending on the stage of the product and design, the ‘KPIs’ I would look out for differ. If the product is launched and a sufficient user base has been established, I’ll track key experience indicators (KEIs) on top of KPIs. Earlier in the design process, it is important to figure out what KEIs are important and why. Who are our target customers? What are the user benefits? KEIs should be specific to the product and user context.

Luky Primadani, Senior UX Designer & Researcher at FlixBus

Some companies tend to focus more on behavioral metrics such as conversion rate and retention rate for example, but not so much attitudinal, like satisfaction or perceived ease of use. NPS should not be the only way to measure the attitudinal metric; perceived ease of use and average satisfaction score could be other metrics to track customer's happiness besides NPS.

Some of them are:

• Conversion Rate,
• Retention,
• Engagement,
• Customer Lifetime Value,
• NPS.

I don't think we have a specific framework to use, but we are in the process of establishing a framework that could be used for both digital and offline experiences.

Joe Allison, Head of UX and Design at Monese

We break Retention into MOM churn, month 3 churn, and month 6 churn, and aim to track and improve on these with new features and design improvements to existing features. On the Revenue side, we measure ARPU - average revenue per active user. We use a system that's loosely based on the HEART framework, tracking a set of user behavior and user emotion metrics. The individual metrics we track are tailored to be relevant for each feature. Emotion includes: sentiment score, ease of use, appearance, trust, loyalty. Behavior includes: task success, time, adoption.

The challenges of measuring design impact in business

The experts’ statements confirm that design is an integral part of impacting the business performance. Setting the right metrics reflecting on business context and goals can empower design teams to show that design has a positive impact on the business's bottom line.