This works aims to provide value to three types of readers. First, for students in statistics, psychology, and the social sciences, I provide a summary and review of three classical statistical methods: factor analysis, principal component analysis (PCA), and probabilistic PCA (PPCA), all of which fall under the category of linear factor models. These methods are widely used in many fields, including psychology, education, and computational biology, and are the cornerstones of many new, more complicated methods. However, most available materials about them are either decades old (and very long and use old-style notations) or cursory. This work provides current coverage of them that is in-depth yet concise.