A statistical criterion is a convention by which certain values are considered relatively probable and others considered relatively improbable. Statistical criteria play a crucial role in the theory of statistics and were originally introduced by Daniel Bernoulli and later independently proposed by Karl Pearson and Ronald Fisher. This article discusses the theory and history of statistical criteria, in particular the density criterion and the distance criterion. Applications for statistical hypothesis generation and testing are discussed. The pedagogical value of statistical criteria is illustrated through a concise and simple explanation of statistical classification. This article also contains discussions on Gauss’ least squares conjecture and Fisher ’s maximum likelihood.