Protecting Against False Inferences: A Comparison Between Stability Controlled Quasi-Experiment and Difference-in-Differences Approaches

Colleen Pinkelman
MAS, 2020
Hazlett, Chad
When a randomized trial is not possible for evaluating the effectiveness of a new treatment, several alternatives have been proposed. Two of these methods are difference-in-differences (DID) analysis and the stability controlled quasi-experiment (SCQE). DID allows for estimation of causal effects with an assumption of “parallel trends”: the trend in average non-treatment outcomes are the same between treated and comparison groups. SCQE relies on an assumption of the outcome's “baseline trend”: the change between cohorts is the same in the overall non-treatment outcome. We compare these two methods under a range of baseline trend assumptions. We also evaluate the methods' reliabilities in protecting against false inferences and over-confidence. Our application is the effect of a placebo health policy, which is known to have no true effect, on 30-day mortality rate among patients treated for heart attack, heart failure, and pneumonia in U.S. hospitals.
2020