Knowing When to Fold 'Em: An Essay on Evaluating the Impact of CEASEFIRE, COMSTAT, and EXILE

Richard A. Berk

Professor Rosenfeld and his colleagues are to be congratulated for their
courage. They have tackled a very controversial and visible set of issues addressed previously by a number of notable criminologists. They have also
chosen to enter the debates with an analysis based solely on observational data. Observational data pose daunting problems when used to draw causal
inferences. At a time when these problems have been thoroughly aired in the statistical literature (e.g., Freedman, 1985; 1987; 1991; Rubin, 1986;
Rosenbaum, 2002; Berk, 2003) and candidly addressed by world-class econometricians (Manski, 1994; Heckman, 2000), the authors of this paper proceed
nevertheless.

One has to wonder, however, whether in this case discretion would have
been the better part of valor. Some empirical problems are just too difficult because of the data likely to be available and the existing methods of analysis. Trying to tease out the impact of the crime control interventions addressed in this paper may be one example.

Over the next several pages I will try to elaborate on this point. In so
doing, I will touch briefly on several statistical concerns about the analyses reported, which may be useful to note, but are of secondary importance. The goal is to focus on some larger issues.

2005-09-01