Edward Snowden’s revelations that the NSA engaged in massive spying operations overseas prompted outrage and political blowback from our closest allies. In an attempt to manage this fallout, President Obama in January 2014 issued Presidential Policy Directive 28 (PPD-28), which established principles to guide signals intelligence activities (basically, electronic surveillance) with an eye toward protecting the “legitimate privacy interests” of foreigners. At the time, civil liberties advocates—including the Brennan Center—praised the general principles that PPD-28 set forth, but cautioned that the devil was very much in the details of how it would be implemented.

There was reason to hope that the Privacy and Civil Liberties Board (PCLOB) would provide those details in its Report to the President on the Implementation of Presidential Policy Directive 28, which was finalized in December 2016 but just became publicly available last week following a Freedom of Information Act request filed by New York Timesreporter Charlie Savage. After all, the PCLOB had conducted an exhaustive investigation into the Foreign Intelligence Surveillance Act (FISA) Section 702 and secured the declassification of an enormous amount of information about how Section 702 operates.

Unfortunately, the PPD-28 report fails to deliver. It adds surprisingly little new information to what was already in the public domain. It focuses largely on the agencies’ official interpretations of the directive, as set forth in their public procedures, without digging in to the actual implementation of these procedures. Its authors seem not to notice discrepancies between the agency policies they describe and the directive. And the report fails to address some of the most open-ended aspects of the directive—the ones for which information about implementation is most necessary. In short, the PPD-28 report is cut from a very different cloth than the PCLOB’s previous major reports, and it falls short of what we should expect from the government’s only independent civil liberties watchdog.

Restrictions on Use of Bulk Collected Information. One of the potentially significant advances contained in PPD-28 is its restrictions on the use of signals intelligence collected in bulk. Under Section 2 of the directive, such data may be used only to detect and counter:

(1) espionage and other threats and activities directed by foreign powers or their intelligence services against the United States and its interests; (2) threats to the United States and its interests from terrorism; (3) threats to the United States and its interests from the development, possession, proliferation, or use of weapons of mass destruction; (4) cybersecurity threats; (5) threats to U.S. or allied Armed Forces or other U.S or allied personnel; and (6) transnational criminal threats, including illicit finance and sanctions evasion related to the other purposes named in this section.

The import of this provision, however, turns in large part on the directive’s definition of “bulk collection.” In a footnote, the directive states that use restrictions on bulk-collected data do not apply “to signals intelligence data that is temporarily acquired to facilitate targeted collection.” This raises a critical question: what do agencies consider to be “temporary” acquisition? For instance, the NSA collected Americans’ phone records in bulk and retained them for 5 years, but it made use of the records only by running queries tied to particular targets. If the NSA were to conduct a similar program involving the bulk collection of electronic communications under Executive Order 12333, would it assert that PPD-28’s use restrictions did not apply? If so, the restrictions would lose much of their potential force.

The footnote was the subject of intense scrutiny and discussion within the civil liberties community; advocates sought clarification from intelligence officials without success. The PCLOB could have obtained the answer, but there is no indication from the report that it even asked the question.

Leaving aside how narrowly or broadly the use restrictions might apply, the PCLOB’s report fails to establish that agencies are adhering to them. With regard to the NSA and CIA, the report simply notes that the agencies “memorialized” PPD-28’s use restrictions in their written procedures. Of course, the directive required the agencies to complete this basic step, and because the written procedures are public, the public already knew they had done so. But the fact that the agencies updated their paperwork as required tells us nothing about how—or even whether—the restrictions are being implemented. It is the agencies’ actions that matter here, not their words.

With regard to the FBI, the PCLOB—again citing the bureau’s written procedures—states that agents must focus their queries on “intelligence information responsive to an intelligence requirement or an authorized law enforcement activity.” The PCLOB seems satisfied with this standard, but it clearly fails to comply with PPD-28. Responsiveness to an intelligence requirement is the bare minimum for foreign intelligence-gathering activities, not a special restriction. Although the requirements themselves (as set forth in the National Intelligence Priorities Framework) are classified, they are purportedly tailored to the threats identified in the Worldwide Threat Assessment of the U.S. Intelligence Community, which is public. A quick glance at that document shows that the threats it identifies—which include “extreme weather,” “disease outbreaks,” and “economic woes” resulting from decreased oil prices—go far beyond the six categories identified in PPD-28. Similarly, “authorized law enforcement activities” would encompass every criminal act within the FBI’s jurisdiction, not the narrow categories of criminal enterprise reflected in PPD-28’s six use restrictions. In short, the FBI isn’t complying with PPD-28 even on paper—a point the PCLOB fails to acknowledge.

Limits on retention and dissemination. Another potentially significant aspect of PPD-28 is its requirement that agencies apply the same limits on retention and dissemination to both U.S. person and non-U.S. person information, “[t]o the maximum extent feasible consistent with the national security.” Civil liberties advocates worried that the caveat could swallow the rule.

And so it did. In their public procedures, the NSA and CIA adopted markedly different standards for the dissemination of U.S. person and non-U.S. person information. The latter could be disseminated if “related to” an intelligence requirement; the former could be disseminated (with identifying information intact) only if “necessary” to understand or assess the foreign intelligence. In other words, the agencies did not interpret PPD-28’s national security caveat as an exception that could be applied on a case-by-case basis; they decided instead that it wiped out the rule, allowing the agencies to apply different standards across the board.

The PCLOB should have inquired whether this approach was consistent with both the letter and spirit of PPD-28. Even if it deemed the approach acceptable, it should have probed the agencies’ determination that applying the same rule to U.S. persons and non-U.S. persons would be inconsistent with national security. What sort of inquiry did the agencies conduct? What facts informed their decision? Was their conclusion not only reasonable but consistent with the directive’s mandate to apply the same standard “to the maximum extent feasible”? There is no such analysis in the report—just one redacted sentence, in each of the paragraphs discussing the differing standards for U.S. persons and non-U.S. persons, that presumably concludes (or states the agencies’ conclusion) that national security considerations warranted the outcome.

With respect to the FBI, the PCLOB simply states the FBI’s position: “FBI contends that all personal information of non-U.S. persons was obtained in the course of a lawful foreign intelligence, counterintelligence, or international terrorism investigation and therefore may be disseminated under PPD-28 and the ODNI guidance.” The PCLOB does not state whether it finds this “contention” to be accurate. Nor does it mention whether the FBI applies the same standard when disseminating U.S. person and non-U.S. person information, as PPD-28 directs, or an entirely different standard. It is hard to imagine a thinner inquiry into implementation of this provision.

On the issue of retention, the report is even more disappointing. PPD-28 states that agencies may retain only information that is determined to meet certain criteria, which should be the same for U.S. persons and non-U.S. persons. It then states: “Information for which no such determination has been made shall not be retained for more than 5 years, unless the DNI expressly determines that continued retention is in the national security interests of the United States.”

With respect to the NSA, the PCLOB states that the NSA was already applying its five-year limitation on retention of U.S. person information to non-U.S. person information. The PCLOB fails to note, however, that the NSA’s five-year limitation, which is set forth in U.S. Signals Intelligence Directive 18 (USSID 18), applies only to “[u]nenciphered communications not thought to contain secret meaning.” USSID 18 clearly contemplates that encrypted communications—which have become increasingly commonplace—may be retained for a longer period. But PPD-28 makes no exception for encrypted communications. How has the NSA resolved this tension? Did the DNI determine that the NSA’s continued retention of all encrypted communications is in the national security interests of the United States? The PCLOB report doesn’t say.

As for the FBI, the PCLOB once again simply recites the bureau’s contention: “FBI indicated that the give-year retention period is not a change in practice for FISA Section 702 data because the FBI’s FISA Section 702 minimization procedures already impose a five-year retention limit for information that has never been reviewed.” In fact, the FBI’s 5-year limit for unreviewed material contains an express exception for encrypted communications. The minimization rules also waive the 5-year limit for communications that have been reviewed, but for which no determination about their foreign intelligence value has been made. This longer retention period for indeterminate communications is plainly inconsistent with PPD-28, under which the absence of any determination about a communication’s intelligence value affirmatively triggers, rather than waives, the 5-year limitation. The PCLOB, however, appears to have simply accepted the FBI’s incomplete characterization of its minimization procedures.

Turning broad principles into practice. One of the real wasted opportunities in this report is its failure to examine how agencies are implementing Section 1 of PPD-28, setting forth “Principles Governing the Collection of Signals Intelligence.” These principles are salutary but not self-executing, and their practical value—as distinguished from whatever symbolic value they may have—depends entirely on how they have been operationalized.

For instance, Section 1 states: “The United States shall not collect signals intelligence for the purpose of suppressing or burdening criticism or dissent, or for disadvantaging persons based on their ethnicity, race, gender, sexual orientation, or religion.” How do agencies ensure that this prohibition is honored? Presumably, agency employees are informed of it during training, and the agencies’ privacy officers review proposed collection activities for compliance. Once the activities are underway, however, who ensures that they adhere to this limitation in practice? What guidelines inform this assessment? If intelligence activities have a significant disparate impact on some ethnicities or religions, is this considered to raise any red flags? Do the agencies make any effort to determine the relative impact of their activities on different ethnicities, races, or religions?

Section 1 also states, “Signals intelligence activities shall be as tailored as feasible.  In determining whether to collect signals intelligence, the United States shall consider the availability of other information, including from diplomatic and public sources.  Such appropriate and feasible alternatives to signals intelligence should be prioritized.” How is this principle put into practice? Is there a requirement, for instance, to identify available alternatives to signals intelligence collection in writing? Is there a process in place for considering those alternatives, culminating in a written determination that is—or at least can be—reviewed at high levels within the agency? Or is this merely an ethos that the agencies believe that their employees already possess? How do the NSA’s current bulk collection programs square with PPD-28’s requirement that collection be “as tailored as feasible”?

Assessing agencies’ adherence to general principles is more difficult than assessing compliance with specific benchmarks like a 5-year retention limit on collected data. Moreover, much of what can be found in Section 1 reiterates principles contained in longstanding agency guidelines or directives. That does not make it less important to review these principles’ implementation. No independent civil liberties oversight body has ever previously examined whether agencies have mechanisms for translating these principles into practice, or whether they remain mere lofty words on a piece of paper. That is exactly the sort of information one would hope the PCLOB would provide.

***

The PCLOB’s PPD-28 report is not without value. It flags the need for guidance on what PPD-28 means by “signals intelligence” so that agencies know when to apply the directive. It reveals that the CIA has opted to be overinclusive in its application of PPD-28 to multi-sourced systems. (Rachel Brand and Elisabeth Collins wrote a separate statement expressing their dismay at the notion that civil liberties protections might be applied where not strictly necessary—a reminder of why PLCOB should be comprised of people with backgrounds in, and demonstrated commitment to, civil liberties.) Perhaps most usefully, it notes the importance of faithful PPD-28 implementation by agencies receiving access to raw signals intelligence for the first time under new 2017 guidelines, and it recommends that those agencies update their practices, procedures, and trainings before receiving such data. None of these contributions, however, substitutes for a robust investigation into how agencies are implementing PPD-28.

I continue to believe that the PCLOB is a vitally important body, and I hope that Congress moves quickly to confirm the remaining two nominees (particularly since the board will otherwise be operating with a significant ideological imbalance). Even though I disagreed with the PCLOB’s recommendations in its Section 702 report, I felt that the board provided a tremendous public service by obtaining the declassification of so much detailed information. In this case, though, the PCLOB seems to have shied away from asking hard questions about what the agencies’ policies looked like in practice, choosing to accept the agency’s own characterizations even when they were plainly wrong. I hope they get back to the gumshoe detective work in their next investigation.

Photo of the NSA’s Utah data collection center on March 17, 2017 by George Frey/Getty Images.