01 Challenge the denominator before the result
Define the eligible population, outcome and time window before calculating a rate.
Analyst interviews when AI can write the query
Challenge the metric. Trace the evidence. Recommend an action.
Investigate the dashboardConversion fell.
What changed?
| Channel | Week 1 | Week 2 |
|---|---|---|
| Direct | 900 | 100 |
| Paid | 100 | 900 |
| Total conversion | 19% | 13.2% |
Practice interviews
02 A valid query can answer the wrong question
Name what one row means. Test a small join before trusting the total.
03 A falling total can hide improving channels
Hold the mix constant before treating a pooled trend as a product defect.
04 The summary is not the investigation
“The method is as much the product as the answer.” (opens the source)
Trace the sample, labels and calculation behind the generated claim.
05 Observed volume is not incremental value
Price the intervention against its counterfactual. State what would change the decision.
Sources
- Amazon, BIE candidate guidance, at the time. Amazon Jobs, “BIE Interview Prep”, Undated. Open the source. Checked against the published text.
- Amazon, BIE candidate guidance, at the time. Amazon Jobs, “BIE Interview Prep”, Undated. Open the source. Checked against the published text.
- Cam Heald, Technical Recruiter — Data Science, at the time. Monzo, “Interviewing for Data Science at Monzo”, Apr 27, 2026. Open the source. Checked against the published text.
- Wren Dougherty, Author, Airbnb Engineering & Data Science, at the time. Airbnb Engineering & Data Science, “Beyond the model: Engineering AI infra with scientific judgement”, Sep 15, 2026. Open the source. Checked against the published text.
- Carlos Sanchez-Martinez, Sean O’Donnell, Lo-Hua Yuan and Yunshan Zhu, Co-authors, Airbnb Engineering & Data Science, at the time. Airbnb Engineering & Data Science, “How Airbnb Measures Listing Lifetime Value”, Apr 1, 2025. Open the source. Checked against the published text.
Last reviewed Nov 10, 2026