Birk: Sectarian Numbers in Iraq

Sectarian Numbers

Guest comment by Joshua Birk

In his September report to Congress, General Petraeus claimed “the number of ethno-sectarian deaths was down by over 55%.” His assessment stands in sharp contrast with the Government Accountability Office report from earlier in the month, which concluded that “It is unclear whether sectarian violence in Iraq has decreased.” The analysis of sectarian violence had been fraught with ill defined, constantly shifting metrics that makes analysis of these numbers difficult and has relied on a pattern of undercounting. This undercounting all but guarantees recent months will always be seen as progress and cast a cloud of doubt over the veracity of the claims of diminished sectarian bloodshed.

The Defense Department periodically issues data on sectarian killings in its reports “Measuring Stability and Security in Iraq.” Comparing the last four of these reports, from October 2006, March 2007, June 2007, and September 2007 can be a maddening exercise which highlights the mutability of statistics on sectarian violence. The reports themselves use shifting terms to describe this violence, chart numbers only in imprecise graphs and frequently disagree on the number of killings which occur in any given month.

However, if you visually measure these reports against each other, a pattern begins to emerge. In the past two months the Pentagon has retroactively increased the numbers of sectarian deaths reported in previous reports. This trend gives rise to the concern that civilian deaths are being reclassified as sectarian killings in order to create the illusion of an improving security situation. If you compare the initial assessment of the post-surge period, June through August in the September ‘07 report, with the initial assessment of the pre-surge period, October through December ‘06, in the March '07 report, only a modest improvement in sectarian violence emerges. General Petraeus’ claim of a 55% drop in ethno-sectarian deaths only emerges when the numbers from October through November sharply increase.

The June 2007 report did not address these concerns and offered no explanation for the retroactively shifting numbers. It was only after facing question about these numbers from reporters that MNFI (Multi-National Force Iraq) Combined Intelligence Operations Center offered an explanation for the changing assessment of sectarian killings. The Iraqi National Command Center, which processed data on these killings, had been overwhelmed with increases in casualties during the fall of 2006. They had developed a tremendous backlog of cases, which were only now being classified as “sectarian murders.”

The September report continued to retroactively inflate numbers of sectarian deaths. In a September 25th article in the Washington Post, Defense Department officials explained the increase as a result of a shifting methodology. In previous reports, they had only calculated deaths that resulted from “murders with distinct sectarian characteristics” but were now charting, “deaths resulting from any sectarian incident.” Broadening the metric they assessed, resulted in another retroactive rise in the level of sectarian deaths. While public discussion of the increases in the September report focused on shifting methodology, the report itself added that the increases were also based on “further data not available for the June 2007 report.”

The lack of distinction between increases based on shifting methodology and increases based on backlogs and unavailable data make these reports exceptionally problematic. The existence of backlogs in particular creates a situation in which recent months will almost always seem to be successful in reducing violence. The illusion of progress may very well disappear by the time that data is processed, but, by that time, we will have a new report, once again incomplete because of backlogged data, and once again showing progress because it only charts a fraction of the sectarian violence in Iraq.

The June report, in which the accounts from February through April radically underestimated sectarian violence, according to the September report, serves as an example of this phenomenon. This undercount is almost certainly the result of unprocessed data, rather than changing methodology. In shifting to calculate “Sectarian Deaths” rather than “Sectarian Murders” the September 2007 report shows an increase of 20% from July 2006 to January 2007. No single month has an increase of more than 30%. That number represents, in rough terms, the impact of the shifting metric. The September report increases numbers of Sectarian deaths in the last months charted in the June report, February through April, by roughly 70%. The vast disparity between this number and the number from previous months suggests a massive undercount.

All of this statistical parsing is necessary to understand the most recent Pentagon reports and to evaluate the claims made about the improving security situation. Undercounts occurred in the June report and there is no reason to assume that the September forecast for the last few months is any more accurate. In fact, given that the report provides data on August, while previous reports stopped tracking two months before the report was issued, the September report may prove to be less accurate than its predecessors. If the undercounts in the September report parallel what we now know about the June report, the level of sectarian violence in Iraq is roughly equivalent to where it was in the summer of last year. This would be an improvement over the horrible chaos and bloodshed of fall and winter, but would fall short of the reduction which General Petraeus has asserted.

Because of the Pentagon’s refusal to release its data, much of this statistical parsing is reading tea leaves. Clarifying the source of these retroactive increases, data which the Department of Defense most certainly possess, would alleviate much of the mystery that surrounds these numbers. The American public has become increasingly doubtful about this war and the military’s constant claims of progress in Iraq. If the Department of Defense wants to reverse that trend, they must become more transparent with the data they release, and the way in which they explain discrepancies in their reports. Until that occurs both the American public and the media should treat all such numbers with skepticism.

Joshua Birk
Assistant Professor
Department of History
Eastern Illinois University.