Civilian Casualties: We Need Better Estimates—Not Just Better Numbers

Over the past year, a swell of voices raised concerns about an apparently waning U.S. government commitment to protect civilians in its military operations. These concerns include:

  • Increasing skepticism about U.S. reported numbers of civilian casualties that are considerably lower than other estimates
  • Both official and independent estimates of civilian casualties increasing significantly in 2017 versus 2016
  • A series of recent high-profile, mass casualty incidents in Mosul, Raqqa, and Yemen
  • U.S. government decisions to decentralize targeting authorities and revisit policies that “exceed the requirements of the Law of Armed Conflict.”

Many have focused on the strikingly low numbers of U.S. official estimates for Iraq and Syria, and how they are unrealistic given the nature of the conflict in 2017: a primarily urban fight waged largely with air-to-ground munitions. Indeed, past studies and reports have discussed the propensity of U.S. estimates to be too low – just as independent estimates can tend to be too high – based on inherent challenges detecting civilian casualties with available military capabilities. The current context of the campaign in Iraq and Syria exacerbates this challenge, featuring attacks on buildings and with few or no boots on the ground. This detection problem means that the U.S. will often not be able to identify when civilian casualties occur based on its own available information, and when it does, there is far from any guarantee that the accounting is complete: the military may, for instance, have full motion video showing several civilians killed but not be aware of dozens more buried in the rubble.

To remedy this situation, many groups recommend steps for the U.S. military to bolster its own capabilities to detect civilian casualties post-strike or to work more closely with independent groups to better consider external information to complement its assessments. And these are good recommendations: certainly in investigations where the U.S. military had a more complete set of information, it has been in a better position to evaluate the effects of its operations.*

However, this focus on improving reporting of civilian casualties may ignore the fact that low reported numbers is a symptom of a bigger problem: the U.S. military has a systemic difficulty anticipating the likelihood and magnitude of civilian casualties when it plans and conducts attacks. This problem in anticipating civilian casualties is seen in cases where the U.S. is surprised by external reports claiming civilian casualties – such as the incident in Mosul in March 2017 where the U.S. later investigated and found it had killed 105 civilians in an airstrike. At the time of the strike, it was unaware of the presence of civilians.

Now, war is an inherently uncertain business. And the U.S. military goes to great lengths to anticipate and avoid civilian casualties, including calculations of collateral damage estimates using population densities, consideration of possible weaponeering solutions, and tactics that reduce civilian harm. While civilian casualties will always happen as an unfortunate consequence of war, there is more the U.S. military can do to anticipate and avoid civilian casualties.

The formal collateral damage estimation process, as rigorous as it is, has never been calibrated with real world data to test its accuracy in predicting operational outcomes. This could be remedied through a study that examines how well estimates match up with actual operational data. Are there particular kinds of situations where these estimates could be improved through a refined model? This can go both ways – such an assessment could show that there are some cases where the model is overestimating anticipated civilian casualties. Improving the model would then improve freedom of action in those cases, without a false concern about civilian casualties. At the same time, if there are cases where the U.S. is systematically underestimating likely civilian casualties, a recalibration of the process could remedy that error.

DOD can also harness advanced technology for improving the accuracy of collateral damage estimates. In 2016, Google made headlines for beating the world champion in the game of Go – a game with more moves than there are atoms in the universe – demonstrating that machine learning can find patterns and opportunities that humans cannot. Google made headlines again recently in its support to Project Maven, a DOD effort to use artificial intelligence to improve exploitation of intelligence feeds for targeting. To help address challenges with civilian protection, Project Maven – and other efforts using artificial intelligence and machine learning – could take on the goal of improving collateral damage estimates and better anticipating the possibility of civilian casualties.

Finally, the ability to anticipate the presence and magnitude of civilian casualties depends on the quality of available data. The military tends to prioritize capabilities that promote targeting effectiveness, such as the ability to positively identify a combatant, and puts fewer resources into capabilities that characterize civilian presence, such as determining pattern of life and other information enabling more accurate determinations of collateral damage estimates. These capabilities include sensors such as Gorgon Stare, which can provide wide area surveillance valuable for detecting the presence and patterns of civilian life, and systems that capture and track civilian information in a common operational picture that can then be queried by those in the targeting process. This information, if captured, can be used to improve the overall process, recalibrating the calculation of collateral damage estimates and exploiting that additional data through artificial intelligence.

While the low U.S. civilian casualty numbers should prompt a rethink on how the U.S. military counts civilian casualties, this is also an opportunity to improve the foundational problem of estimating possible civilian casualties prior to the use of force. We have offered specific actions the U.S. military can take to improve this estimation process overall. And this is a virtuous circle: better anticipating civilian casualties on the front end can yield fewer incidents that need to be accounted for after the fact, improving numbers and, even more importantly, saving lives.

Of significance, this virtuous circle works in both directions. Better estimating the number of civilian casualties post-strike should improve pre-strike decisions. Commanders will have a more reliable understanding of the expected damage of US operations over time. Unfortunately, this also means if DOD post-strike civilian casualty estimates are generally too low, presumably collateral damage assessments will also have generally underestimated the likelihood and magnitude of civilian casualties pre-strike (a vicious circle).

None of this is to say that recent or ongoing strikes exceed the proportionality analysis required by the laws of war. That’s not the point. The DOD is committed to the goal of minimizing civilian casualties. We offer this analysis as a pathway to better reach that objective.


[*] Sarah Sewall and Larry Lewis, Joint Civilian Casualty Study, Joint Staff, 2010. 

About the Author(s)

Larry Lewis

Director of the Center for Autonomy and Artificial Intelligence at Center for Naval Analyses. Lewis spent a decade analyzing real world operations as the project lead and primary author for many of the Department of Defense 's Joint Lessons Learned studies.

Ryan Goodman

Co-Editor-in-Chief of Just Security, Anne and Joel Ehrenkranz Professor of Law at New York University School of Law, former Special Counsel to the General Counsel of the Department of Defense (2015-2016) Follow him on Twitter @rgoodlaw.