Deep learning techniques have become commonplace tools for complex prediction, classification, and recognition tasks. Yet, the performance of such learning techniques is highly influenced by user-set hyperparameters. As a result, efficient hyperparameter tuning and optimization is an increasingly important area of study. Traditional model-free tuning methods are often computationally inefficient and may miss optimal settings, while model-based approaches rely on parametric models and cannot easily be parallelized. In this thesis, we propose the use of experimental design techniques, a Design of Experiments (DOE) Approach, to more efficiently and intuitively optimize hyperparameters. We use fractional factorial designs, nearly orthogonal arrays, sliced Latin hypercube designs, and composite variations of these three small-run designs to identify relationships between continuous, discrete and categorical hyperparameters and test accuracy in convolutional neural networks using beta regression. We find that our proposed methodology successfully identifies optimal hyperparameter settings for convolutional neural networks trained on the MNIST dataset.