I will apply the GAME framework to answer Instacart product metric interview questions.
Interviewer: Walks in the room. After the initial introductions, “ I would like to discuss product metrics”
You: Sure! (Internally you are just hoping that the problem is a easy one :))
Interviewer: Let’s start by designing a metric to measure the success of Instacart. How would you go about defining the metric?
You: OK. Here is what I would do
Most of the Product manager interviews usually include a section on metrics. One of the most important ways to tackle metric questions is to pick a framework and tackle the most important product metric.
A detailed explantion of the GAME framework is provided in another blog topic, Answering Product Metrics Interview Questions with GAME Framework. You should read that blog post to get a good understanding of the GAME framework for metric measurement.
GAME stands for,
Start with defining the product or the feature.
Instacart is a grocery delivery business. The goal of the company would be to deliver groceries to the end-user with convenience in mind to reduce the frictions involved in physical shopping experience. Instacart customers log in through a mobile or web app, select and purchase their groceries, which are delivered to their homes. On the back-end, Instacart finds gig workers who perform the shopping and delivery of the groceries.
Instacart includes buyers that order the products, retailers, or grocery stores that supply the requested products and the shoppers who work as freelancers to deliver the grocery. There are several metrics that we could check starting with the buyer metrics or supply-side metrics. For the purpose of defining the solution space within an hour, we will stick to buyer metrics for this question.
Before jumping into the solution space, I will confirm that the interviewer agrees with the specific Goal that I selected for the success of the product. The next step is to define the Action.
What are the actions that a user will take? The user will use a mobile or a web app to identify the store that they need the groceries from. Then they add the items that they want to the shopping cart. After adding the items the customer would check out and choose the delivery window for the shopper to deliver the items.
Once you have a prioritized list of actions, it’s time to define associated metrics for each one. In this step, it’s important that you define the metrics that you will measure with precision.
We had picked buyer side engagement as the Goal and the actions taken by the user earlier.
We will define FOUR metrics for this goal
- Adoption: How many new users signed up for Instacart over a period of one month
- Retention: How many existing users place recurring orders over a period of one month
- Engagement: Average number of time spent on the site, the average number of products adder per order, user discovering new stores and browsing products
- Satisfaction: Positive feedback and net promoter score.
Generally, revenue is NOT a recommended goal. Companies that focus solely on revenue fail to consider that user experience is a more important health metric for the overall company. First and foremost, deliver an incredible and worthwhile user experience. Then, you can consider monetization metrics.
The last step in the metric process is Evaluations. I will conclude the interview by evaluating the metrics chosen. I will also highlight the trade-offs and limitations.
- Adoption: Instacart is an established startup focused on relentless growth and expansion. Part of the growth model is to acquire new customers every month and grow their customer base. We need a metric to measure the signup rate to keep track of the growth. Measuring the number of new users signed up over a period of a month is a good metric for tracking growth.
- Retention: Instacart has several high profile competitors such as Amazon Prime. A key metric for Instacart is recurring orders from existing customers. Measuring user retention is also a critical success metric for the company growth.
- Engagement: This metric measures the level of user involvement with the product or feature, over some time period. Engagement is an important score for determining if the product features and placement are engaging for the customers to pay attention, stay engaged, and also discover new products and features introduced.
- Satisfaction: Instacart needs to measure customer satisfaction scores to ensure that customers are happy with the ordering process, orders fulfilled, and the time of delivery. Low satisfaction would lead to lower retention. Satisfaction scores also help product managers discover issues in the supply chain.
The last key step is to show that you understand the strengths and weaknesses of your recommended metrics, by listing the tradeoffs or limitations. Measuring satisfaction is hard since the user mainly leaves feedback if they have an issue. Feedback will be highly weighted towards the negative. The metrics measured will not provide the supply side of the retailer metrics and also excludes the revenue metrics.
Always use a framework to answer metric questions. Do not jump into solution space immediately.
In this blog, we used the GAME framework to answer the Instacart product metric question. I have outlined a step-by-step method you can use to solve metric questions. I encourage you to try solving a couple of the sample questions using the steps defined in this blog.