Startling recent claims and evidence that armed drones are less accurate than piloted aircraft have prompted us to investigate the numbers behind them. We do so in a piece entitled, The Precision of Drones: Problems with the New Data and New Claims, which can be found at E-International Relations. Although Just Security is not always focused on the empirical research of such questions, we have observed that an alternative disciplinary view is often welcomed by its readers.
Specifically, we analyze the data found in the Director of National Intelligence report released in July on “U.S. Counterterrorism Strikes Outside Areas of Active Hostilities” and a recent article by Micah Zenko and Amelia May Wolf which claims that “Drones Kill More Civilians Than Pilots Do.” The conclusion of the latter analysis is clear in its title, whereas the newly released U.S. data show that operations that rely primarily on armed drones (in Pakistan, Somalia, and Yemen) kill over 59 times more civilians per strike when compared to the air campaigns in Iraq and Syria (based on the most recent numbers supplied by the Pentagon). These numbers appear to fly in the face of assurances by U.S. officials, including the President, that drones are more precise than the alternatives.
While some aspects of this newly available data are undeniably troubling for those working off the assumption of an increased accuracy with drones, it has often been misread and misrepresented and its implications exaggerated. In fact, the most recent evidence also provides some support for the U.S. administration’s claim about the relative precision of drones (as will be shown below). As a result, we argue that it is too early to draw definitive conclusions, as the data available is simply too sparse, unreliable and contradictory. Claims about drones’ tactical precision, whether for or against, should not be used to side-step the host of other legitimate worries about their use, especially outside areas of active hostilities—such as whether they violate the law on the use of force, human rights, or promote terrorist recruitment.
What follows is a brief overview of the argument and evidence presented in our article. Furthermore, we supplement our analysis with some additional reasons to be skeptical of how some commentators interpret the existing data.
The most general problem with the case for the imprecision of drones is that it is largely based on a comparison between the air strikes taking place in Iraq and Syria (predominantly by piloted aircraft) and the air campaigns in Pakistan, Somalia and Yemen (employing mostly drones). However, as neither set of these campaigns is carried out by only one type of aircraft, both of them end up making poor proxies. The official data provides no details on what weapons platform caused specific civilian casualties and thus masks the most important facts for analysis. As just one example, in 2009 a cruise missile launched from a ship killed 41 civilians in Yemen according to Human Rights Watch. If the campaigns are used as proxies, these civilian deaths would be wrongly attributed to drones which would distort the analysis of precision.
There are also issues with the utility of the data that officials have provided. These include: the DNI data on strikes “Outside Areas of Active Hostilities” does not break the numbers out by country, or even specify which countries are covered (do the numbers include Libya or all parts of Pakistan?); this data also cannot be broken down in terms of time and therefore obscures important differences of performance (including improvement over time); and none of the official data provides figures on “signature strikes” (i.e., their frequency or whether they are used more in some theaters than in others), nor does the official information supply the criteria (i.e., patterns of life) employed to distinguish legitimate targets from civilians, which raises questions concerning the accuracy of counts of civilians killed.
In addition, the Pentagon reporting on Iraq and Syria appears to suffer from a significant time lag in acknowledging civilian casualties, which currently makes this air campaign look astonishingly accurate. Of course, this may not be borne out. While the United States presently acknowledges only 55 civilian casualties from coalition airstrikes in Iraq and Syria, the independent database Airwars currently reports 1595.
This also raises the issue that non-governmental organizations may have more access and ability to identify civilian casualties in non-battlefield settings, which could favor higher reported numbers in those areas. Similarly, U.S. intelligence information could be better in some countries than others—leading to more accurate targeting and greater opportunities for strikes with fewer civilians around. Some areas of operation may also involve more dynamic rather than deliberate preplanned airstrikes, which should affect the accuracy of targeting and numbers of civilian casualties. It may also be the case that certain areas involve higher-value military targets, which could entail a greater allowance for anticipated civilian casualties in military decision-making. The point is that all these factors may vary according to the very same crude proxies used to distinguish between manned and unmanned aircraft, and there’s no easy way to control for these variables.
Finally, and perhaps most noticeably to the readers here, discerning between civilians and fighters is an intensely contested legal question within international law that lies at the heart of non-international armed conflict. We know it is internationally recognized that those directly participating in hostilities can be attacked, yet removing the temporal component to target individuals based solely on membership status with an armed group or including those who have a “continuous combat function” away from hot battlefields is more controversial. Thus this issue again muddles the tallying of civilians vs. combatants since there is little agreement over the categories either before or after a strike.
In addition to identifying and illuminating these problems with the official data, we also looked into the difficulties behind the recent claim by Zenko & Wolf that “Drones Kill More Civilians Than Pilots Do.” Three stood out:
- they use, as the basis of their comparison, “bombs and missiles dropped” in Iraq and Syria, and compare these to drone “strikes” in Pakistan, Yemen and Somalia. However, drone strikes often involve the firing of many missiles and so this amounts to comparing apples to crates of apples;
- they incorporate the exceptionally low Pentagon (DoD) count of civilian casualties recognized to date only on one side of the ledger: in Iraq and Syria. This choice skews the results toward relatively higher civilian casualty counts in other campaigns; and
- they compare across two importantly different time periods: U.S. airstrikes in Iraq and Syria which began less than two years ago, against over seven years of drone strikes. This conceals the fact that the precision of drone strikes has improved steeply over the period, with the vast majority of civilian casualties in the early Obama years.
In view of these shortcomings, we provide an alternative analysis of the available numbers to conclude:
If the campaigns are compared on the same base of measurement (by strike), without the distortion produced by introducing DoD civilian casualties on one side only, and over this same time period (2014-2016), drones appear to be a little more precise. In other words, the rate of civilian casualties per strikes is 33% higher in Iraq and Syria, where manned strikes predominate.
These findings are confirmed by examination of data from Afghanistan and Pakistan over the same period, which show that drone strikes in Pakistan have been somewhat more precise than the mixed manned and unmanned strikes in Afghanistan. Moreover, although drones have flown the majority of recent missions in Afghanistan, the majority of civilian casualties have been caused by manned aircraft.
So although the data we have examined does not vindicate the government’s declarations of the extraordinary precision of armed drones, it does cast doubt on the proposition that this new weaponry has actually been less accurate than piloted aircraft. Of course, this is not likely to be a surprising conclusion. When we think about drones’ unique capabilities—a fixed visual on the target for the crew, extended loitering times, combined with small laser-guided munitions—it seems natural to expect the relative precision of drones before the statistical evidence becomes available.
Nevertheless, this discussion addresses just one side of the multifaceted questions concerning the effectiveness of drones—that of tactical efficacy. We must be cautious not to confuse killing one category of leaders with automatically defeating an enemy group, bringing an end to retaliation, or achieving the long-term strategic goals of a nation—such as constructing a stable international system. All of this is to say that the issue of drones’ accuracy is an important part of the larger empirical puzzle, and we should be careful with our readings of the available data.