# Mailing holdout setup and reporting checklist

A complete planning form for eligibility, assignment, suppression, measurement, exceptions, and the final decision.

## Decision and ownership
- Decision the pilot will inform:
- Campaign owner:
- Analyst and outcome-report owner:
- Mailing operations owner:
- Budget ceiling:
- Minimum business improvement worth acting on:

## Eligible audience, before assignment
- Inclusion rule and export timestamp:
- Exclusion rules, including contact preferences:
- Unit of assignment and analysis (person, household, or account):
- Duplicates resolved before assignment:
- Eligible count:
- Restricted location of the source list:

## Assignment and protection
- Random assignment method, seed or reproducible record:
- Planned mailed and holdout counts:
- Frozen assignment file and version:
- Holdout suppression location:
- Owner who verifies that manual resends honor the holdout:
- Ordinary service and support communications retained:
- Other campaigns received by each group:

## Outcome and timing
- One primary outcome, with qualification and duplicate rules:
- System of record and query version:
- Planned send date:
- Identical observation start and end:
- Analysis date and stopping rule:
- Sample-size and uncertainty method reviewed by:

## Execution log
- Creative and list versions approved:
- Attempted, accepted, failed, and returned mail recorded separately:
- Unexpected crossovers or other interventions:
- Deviations from the original plan and reasons:

## Final report
- Assigned units and qualifying outcomes in each group:
- Outcome rates with explicit denominators:
- Absolute percentage-point difference:
- Relative difference, or undefined when the control rate is zero:
- Estimated additional outcomes and uncertainty analysis:
- Complete mailing cost and contribution assumptions:
- Decision, owner, and bounded next step:

Keep recipient-level data in your restricted workspace. This blank template contains no customer records. The primary comparison retains original assignment; any secondary subset analysis must be separately labeled.

---

From Scribble: https://scribblecards.com/blog/card-mailing-holdout-worksheet

Editorial review: 2026-09-16

## Sources

- NIST/SEMATECH e-Handbook of Statistical Methods: [How do you select an experimental design?](https://www.itl.nist.gov/div898/handbook/pri/section3/pri33.htm), accessed 2026-09-16.
- NIST/SEMATECH e-Handbook of Statistical Methods: [Completely randomized designs](https://www.itl.nist.gov/div898/handbook/pri/section3/pri331.htm), accessed 2026-09-16.