We consider the problem of learning the structure of the factor analysis model. The traditional method of Exploratory Factor Analysis (EFA), despite its widespread application, is often criticized for its ad-hoc use of rotation criteria for learning solutions. Additionally, more recently developed penalized EFA methods partially address these issues, but remain computationally intense. We propose a fast correlation thresholding algorithm, that is theoretically motivated by graph theory, to simultaneously learn the structure of a factor analysis model for an unknown number of factors. We derive the conditions for structural identifiability and parameter uniqueness, as well as show asymptotic consistency for our algorithm. Finally, we present a simulation study and real data example to test and demonstrate its performance