Type I Error (False Positive)

Short answer

A Type I error, or false positive, is wrongly concluding that a test variant had a real effect when the observed difference was actually due to chance. The significance level sets its expected rate; a 5% threshold accepts roughly a one in twenty chance under the null hypothesis. Practices like peeking increase it.