This is PART 2 of the product metric blog series. Please read How to define and measure the metrics of a product (PART 1) that lists the top metrics needed for a product manager, before jumping to PART 2.
How do we create a framework to identify, categorize, and decide on the final metrics for a product, that also ties back to the business goals. There are different frameworks available to define success metrics. I have picked the top three metrics below to measure the success metrics of your product or feature.
FRAMEWORK 1: HEART FRAMEWORK
The first one is the **HEART** framework. HEART (developed at Google) stands for Happiness, Engagement, Adoption and Retention, and Task Success. This framework defines UX metrics of the product.
- Happiness: This index measures user attitudes, often collected via survey. For example satisfaction, perceived ease of use, and net-promoter score.
- Engagement: This index measures the level of user involvement with the product or feature, over some time. Examples might include the number of visits per user per week or the number of photos uploaded per user per day.
- Adoption: This index measures the number of new users of the product. For example the number of accounts created in the last seven days or the percentage of Gmail users who use labels.
- Retention: This index measures the rate at which existing users are returning. For example: how many of the active users from a given time are still present in some later time?
- Task success: this includes traditional behavioral metrics of user experience, such as efficiency (e.g. time to complete a task), effectiveness (e.g. percent of tasks completed), and error rate. This category is most applicable to areas of your product that are very task-focused, such as search or an upload flow.
As a PM, you don’t metrics to hit all the different categories in the HEART framework. Instead, you should choose the metrics that are the most important for your project. The HEART framework can help you decide whether to include or exclude a particular category. These can be applied at a number of levels — from the whole product to a specific feature.
FRAMEWORK 2: AARRR Metrics
Dave McClure defined a framework for startups to define the success metrics of their product or features. The metric framework was titled AARRR. AARRR is a framework focusing on the company growth and therefore it became widely accepted in the startup world. This framework also applies to bigger companies though.
The gist of the framework is a metrics funnel starting at the acquisition phase and moving all the way to the revenue phase.
- Acquisition: where / what channels do users come from? (e.g. # of app downloads, click-through rate from pay-per-click ads)
- Activation: what % of users have a “happy” initial experience? (e.g. Sign-up rate, subscription rate)
- Retention: do they come back over time? (e.g. monthly active users)
- Referral: do they like it enough to tell their friends? (e.g. # of new sign up through referral)
- Revenue: can you monetize any of this behavior? (e.g. Conversation rate)
FRAMEWORK 3: RARRA Metrics
This is identical to the AARRR metric seen earlier. The only difference is that the Retention gets higher priority over Acquisition. The rationale is that the product team should first focus on retaining the user base by providing great user value, instead of acquiring new customers.
- Retention: Give your customers incredible value so they will use your app over and over.
- Activation: Make sure new users see your app’s value the very first time they launch it.
- Referral: Get them sharing and talking about your app.
- Revenue: How to monetize your app
- Acquisition: Acquire new users
As a PM you need to decide and pick the framework that best suits your product or feature.
What should be the end goal of product metric measurements?: Product metric measurements can be used to confirm or modify any assumptions that you have about the new product release or features planned. Once confirmed with metrics the next step is to change the product roadmap or plan.
Any change in product roadmap should be communicated to all the stakeholders including the executive staff. Finally, once the stakeholders are aligned, you should reconvene with the development teams to discuss and execute the new plan and strategy.
Metrics should be included for every release of the product or feature and the cycle should be repeated as a feedback loop.