Whereas making ready for my Product Information Scientist interview, I scoured the online for ideas and frameworks to deal with the “Success Metrics” interview query. I discovered bits and items however no full end-to-end information. Therefore, I’m excited to share the last word framework I created throughout my preparation, which helped me get a suggestion from Meta. Do have a look and I hope it helps you too.
Framework — Assuming you might be a part of the DS workforce for the Fb group, how would you outline your metrics for achievement?
Clarifying the query — At all times begin by asking clarifying questions. Clarify each phrase of your query and most significantly, the product in query intimately. Not asking questions is a certain crimson flag.
Let me ask a number of clarifying questions: Are you speaking about teams within the Fb core app? Am I right in understanding that teams could be personal or public?
After you have a transparent query, take a deep breath and begin answering it. Make an enormous detourSure, that is proper. Do not reply the query straight away. It is most necessary and anticipated that you just speak about your product, your organization mission, and the way the 2 match collectively at the start. So be sure you speak in regards to the following:
Let’s begin the reply with an enormous detour: As a substitute of leaping proper in and answering the query, let’s begin by speaking about your product, your organization’s mission, and the way the 2 match collectively.
Firm Mission
Before everything, Meta’s mission is to convey folks nearer collectively and empower them to construct neighborhood.
What’s the product objective and the way does it connect with the corporate’s mission?
FB Teams is an important product for Fb as a result of its objective is to convey folks with frequent pursuits nearer collectively, and that objective ties into Meta’s general mission of bringing folks nearer collectively.
person — At all times speak in regards to the person. Almost each product has two person sides: producer and client. It is crucial to speak about each person journeys, and much more necessary to supply metrics that cowl each side later in your reply.
A FB group has two customers: the group’s admins who’re producers, and the group’s members who’re shoppers.
Admins create teams, resolve if the group must be public or personal, ship invites to customers to hitch, publish hyperlinks/media/info and begin conversations throughout the group.
Members can see it of their feed, seek for it, or be invited to hitch the group (relying on whether or not the group is public or personal) and as soon as they be part of they’ll contribute to the group by posts, feedback, likes, shares, and many others.
Advantages + Prices (for each customers and companies) — Earlier than we get into the metrics, it’s a good suggestion to briefly focus on the advantages and prices of the product for each customers and firms.
One of many primary advantages of FB Teams is that they carry collectively customers with frequent pursuits, which ties into Meta’s mission and contributes to engagement throughout the FB app. FB Teams additionally enable Meta to higher perceive customers’ pursuits by wanting on the teams they belong to, which in flip helps create higher suggestions and a extra participating feed for customers.
Alternatively, FB Teams might lower person engagement with the FB Newsfeed, which is the “coronary heart” of the FB app and the place it generates income by adverts. One other potential disadvantage is that if a gaggle doesn’t have sufficient members or engagement, admins might develop into discouraged and create “hole” teams.
What forms of metrics do you have to have a look at? — Now it’s time to begin speaking metrics. Select two to deal with: acquisition/activation/retention/engagement/monetization.
Now, concerning metrics, since FB Teams is a mature product, I feel it is smart to deal with engagement + retention.
Checklist your organization’s NSM (North Star Metric) + reported figures — When selecting key metrics, it’s necessary to maintain the NSM and reporting figures in thoughts.
Earlier than we dive into the success metrics of the product, let’s briefly speak about Meta’s NSM (every day classes per person). Moreover, Meta additionally reviews on Wall Road DAU and MAUS. So, when speaking about success metrics, it is necessary to maintain the above factors in thoughts, particularly when selecting your major metric.
Metrics — Now it is time to present metrics. I’ll present metrics from the 2 areas of focus talked about above, however it’s necessary to not go overboard. Two or three key metrics per space must be sufficient. Notice that I’ve metrics that cowl each the producer and client facet of the person.
You will need to have metrics that cowl each the producer and client sides of customers.
engagement:
Variety of teams created per week with not less than 3 members per person
Variety of in-group interactions per person per week
Variety of classes involving teams per person per week
Retention:
Variety of lively days per person within the final 7 days (lively = FB teams used)
Week 2 Retention = Variety of customers who have been lively not less than as soon as per week for 2 consecutive weeks / Variety of customers who have been lively solely within the first week
**I selected weeks because the time interval as a result of I consider FB teams are usually not meant for every day use.
Choose Driver/Secondary/Guardrail metrics — Now, select your major, secondary and guardrail metrics from the record of metrics above. Remember to debate trade-offs.
Driver Metrics:
Variety of group classes per week
[# of sessions is easy to explain, captures users behavior and ties with Meta’s NSM. If sessions involving groups are up then more interaction among users → creating communities.]
Secondary Indicators:
Variety of teams created per week with not less than 3 members per person (To make sure provide)
Variety of in-group interactions per person per week (To seize demand)
Variety of customers lively for two consecutive weeks (this week + final week) / Variety of customers lively final week (2nd week retention)
Guardrail Metrics:
Variety of customers faraway from a gaggle
Variety of teams closed or reported
Variety of offensive/inappropriate posts posted within the group per week
Time spent within the information feed (we do not need customers to cease utilizing the information feed)
abstract — It is at all times a good suggestion to run by the story you have created and present how/why it solutions the query.
That is the framework that labored for me and helped me safe a suggestion from Meta. You should utilize the identical framework for any product/success metric query requested in a Information Science function. It is necessary to have a well-constructed reply that completely explores all the important thing parts and leads the interviewer by your thought course of. I hope you loved it and would love your suggestions within the feedback under.
