The objective of this research is to apply techniques of randomized controlled experimentation including experimental design and analysis in user experience optimization and mobile app development. Online controlled experiments started to be used in the late 1990s with the growth of the Internet. Nowadays, many Internet companies leverage controlled experiments, especially A/B testing, to understand and make decisions at every step of product development. Large sites, including Facebook, Google, and Airbnb, run hundreds of A/B testing every month on features from UX design to algorithms for growth. The power of running controlled experiments is the ability to establish causal inference and quickly validate new product ideas with statistical evidence of success. This process can ultimately help organizations, especially Agile software, optimize products with iterations by better understanding the corresponding impact on the user experience and recognizing the best performer from a list of variations.