Skip to content
The Madison Avenue Field GuideA field guide to marketing & advertising, by Brandon Mina

GlossaryMeasurement & AttributionFrequentist Testing

Measurement & Attribution

Frequentist Testing

The classic hypothesis-testing approach that judges results by p-values and confidence intervals, usually requiring a fixed sample size set in advance.

By Brandon MinaCEO, BrandPilot AIUpdated November 2026

Short answer

Frequentist testing is the classic hypothesis-testing approach to experiments, judging results by p-values and confidence intervals. It typically requires setting a fixed sample size in advance and avoiding early stopping, since repeatedly checking results inflates false positives. It is the most widely used framework in A/B testing tools.