The hardest question in migration is: “is this hurting retention or not?” The honest answer requires a comparison group — subscribers who got the migration offer, vs subscribers who didn’t, with everything else (tenure, plan, geo, engagement) held constant. That’s a matched IAP holdout.Documentation Index
Fetch the complete documentation index at: https://recurr.dev/docs/llms.txt
Use this file to discover all available pages before exploring further.
What the holdout is
For each migration wave, a parallel set of subscribers is reserved as a control:- Same tenure profile (e.g., 12+ months on subscription)
- Same plan tier (annual or monthly, premium or basic)
- Same engagement quartile (heavy / mid / light usage)
- Same geo bucket (primary geos only, or matched mix)
- Random assignment within the matched stratum
How it’s used
Every wave’s report compares the migrated cohort and the holdout on:| Metric | What it tells you |
|---|---|
| 30-day retention | Did the migration touchpoint cause near-term churn? |
| 60-day retention | Did the renewal cycle on web hold compared to IAP? |
| Refund / chargeback rate | Did the web checkout introduce billing disputes? |
| Support ticket volume | Did subscribers find the migration confusing? |
| Reactivation rate | Did churned subscribers come back at the usual rate? |
How big is the holdout?
Right-sized for statistical power, not 50/50. A typical wave reserves 15–25% of the matched cohort as holdout — enough to detect a 2-percentage-point churn delta with reasonable confidence, small enough that the migration captures most of the cohort’s recoverable margin. Precise sizing depends on the cohort’s baseline churn variance. Higher-variance cohorts need larger holdouts; long-tenure low-churn cohorts can run with smaller holdouts.Why this matters for the buyer
A migration without a holdout is faith-based. You can see migration rate, but you can’t see whether it cost you anything in churn. That’s not a defensible answer to a finance question — “is the recovered margin net positive after retention impact?” With a holdout, the question has a number. Pilot wave 1 finishes with a defensible migration recovery − retention cost result, and wave 2 ships only if that result clears the agreed bar.The holdout doesn’t disappear — those subscribers stay on IAP indefinitely if they want. They simply weren’t part of this wave’s offer. They can be folded into a later wave’s cohort once that wave’s spec is set.