Prediction in Multilevel Models
Advisor: Jan de Leeuw
Multilevel modeling is an increasingly popular technique for analyzing hierarchical data. Moreover, many practical problems are readily amenable to multilevel model prediction, e.g., educational data where students are nested within schools. We consider the problem of predicting a future observable $y/sb[*j]$ in the jth group of a hierarchical dataset. Three prediction rules are presented and assessed via a Monte Carlo study that extensively covers both the sample size and parameter space. Specifically, the sample size space concerns the various combinations of level-1 and level-2 sample sizes, while the parameter space concerns different intraclass correlation values. The ideal case, which is initially examined, is subsequently modified in order to better approximate situations actually encountered in practice: First, following the work of Harville (1985), the prediction problem is embedded in a prediction error decomposition framework with respect to both parameters and data (both level-1 and level-2 data). Second, the effect of model mis-specification on prediction is examined, thereby accounting for the model uncertainty that is certain to surface in practice. Third, the sensitivity of the prediction rules to out-of-range predictions is studied. Indeed, it is often these cases which prove most interesting in practice. Thus, from the initial prediction problem, we proceed down the following path: (1) change of available parametric information, (2) change of available data information, (3) change of model specification, and (4) change of the process to be predicted. Finally, employing a sub-sample from the National Educational Longitudinal Study of 1988 (NELS:88), we illustrate the lessons learned and provide an approach to predicting future observables, thereby facilitating the difficult problem of multilevel model specification.