The 5 steps of better product design: 5. Measure impact

Use mobile analytics to define, collect and format live data from your application to validate your hypotheses, monitor user behaviour and measure your product’s success.

This data can enable you to gather large-scale insights into how your products are being used, which can be viewed alongside qualitative research to guide your product decisions.

This is step 5 in a 5-piece series exploring the stages of product design.

As discussed in the ‘understand’ phase of this series, there are two types of research that can provide useful user insight; qualitative (customer interviews, focus groups, app store reviews, customer support feedback) and quantitative (survey results, app store ratings, data analytics, crash reports).

This post will focus on analysing quantitative data post-release.

Tips for product designers

Here are some tips to help you get the most from your analytics, work out what to track and plan for the future.

Stick to your goals

It’s important to stay focussed on the main goals of your release. What are your overall KPIs? What is your hypothesis? How do you define product success? What do you need to track to confidently measure success?

Think big

When working on a specific KPI, it’s easy to lose sight of the bigger picture. Make sure you communicate product changes across the company prior to each release and be aware of how product changes might affect user behaviour in other areas.

Define early

Defining your analytics requirements early on will help you ensure that tracking points are implemented in time for release with the appropriate tracking tools. Consider questions that might arise post-release and ensure you have sufficient data to provide validated answers.

Be specific

It’s important to specify exactly what you’re measuring. Start with a question, define a measurement and hypothesise possible outcomes. This will help you to define the best tracking metrics for your needs and generate valuable insights.

Start small

When testing a feature for the first time, limit the release to a small percentage of users. This will allow you to measure the impact and react quickly.

Be transparent

Taking a learning-based approach to product development can help your team to understand the impact of product changes more clearly. Holding regular progress reviews can also encourage a unified focus and an increased feeling of responsibility across the team.

What can we track?

If you’re a product designer, you’ll no doubt spend a lot of your time focussing on business goals and customer needs. On release, you need a way to track whether you have succeeded or not. Some general measurements can be useful, such as customer satisfaction or number of downloads, but you may also require more detailed metrics. The degree of granularity will depend on what you’re trying to understand.

The most important thing is to ask the right questions. Why are you implementing this change? What are you trying to achieve? What is success and what is failure?

Here are some tracking options to consider:

User satisfaction

A simple in-app questionnaire can help you to benchmark an overall user satisfaction rating, measuring customer happiness.

Click interaction

Check your analytics for the percentage of users that are clicking on important elements (eg buttons, links, images, icons). This might help you understand the value of the action or the visual prominence of the element.

Screen views

Find out the percentage of users that are landing on a particular screen. Tracking this across multiple screens will help you understand how far users proceed through an expected user journey and identify any major drop-off points.

Load time

Check how long your application, or a specific screen, is taking to load. This can help you to identify issues and optimise your loading times.

Sessions

Monitor the number of overall sessions and sessions per user. This metric can help you measure product growth over time and the level of customer retention.

Session duration

Find out how long users typically spend using your application. This can give you an indication of levels of user engagement.

Account status

Check how many of your users sign in when they use your application and take a look at any demographic information you hold on those users. This can help you assess the value of any signed-in-only features and which user groups they appeal to.

Entry points

Take a look at where your users are coming from. How many users are opening the app directly from their device versus a web smart banner, link or native notification? This can help to analyse and optimise retention and measure the effect of notifications.

Crash reports

It’s always a good idea to keep track of the number of crash reports and keep an eye on problem areas. This can help you improve your overall crash rate and increase user satisfaction.

Conclusion

Analytics tracking allows you to measure the impact your products are having on the business and your end users. By asking the right questions, you can validate hypotheses on a large scale and gather insights to help you optimise your future product development. It can also create a unified focus across the team and generate clear success indicators for stakeholders.

I hope you’ve found this series interesting and taken away some useful ideas for your own projects. If you have any other tips or feedback you’d like to share on this subject then please get in touch on leonie@novoda.com.

Originally published at https://blog.novoda.com on February 14, 2017.