Rethinking the Intelligence Community in the Golden Age of Open Source Intelligence

By Danny Lam

Competition between public and controlled information is as old as life.

From the beginning of time, there were information hoarder / controllers and disseminators / distributors.   Information science tells us that 99% of information is public and only a tiny portion is significantly restricted.

The explosion and spread of technologies from the 19th century onwards have favored disseminators.  Tool and techniques to collect and archive primary data like voice recorders, cameras, etc. are now ubiquitous.

Communications networks have expanded both coverage and bandwidth to a level unimaginable at the dawn of the industrial age.

What remains slow changing and scarce are those with the skill and can make sense of the raw data, search, manipulate, and organize it to elicit facts and actionable intelligence from a body of information “out there.”

Automation of these functions is presently one of the fastest growing technology areas, i.e. rather than manually review surveillance videos, automation techniques like facial recognition allow near instant identification and tracking of persons.

This is now branded as “artificial intelligence.”

Collection of large amounts of data is a capability that security agencies, particularly in totalitarian regimes, have seized on and developed world class transnational systems for their purposes.

The amount of data collected on most individuals in developed economies far exceed what the best cold war era agencies like STASI could have dreamed of.

Making sense of this vast trove of data, is another matter.

Traditionally, intelligence bureaucracies relied on professionals to collect, archive, analyze, and make sense of the data.

The US have long maintained a lead in this area from the first days of the Office of Strategic Services to present in areas like signal intelligence and its cognates like encryption, analysis, etc.

The US is a world leader in high tech means to collect data that began with difficult to intercept aircraft like the U-2, the ultra-fast SR-71, and then satellites like Corona that obtained data no other nation have.

Over time, the SIGINT capability of the US and allies grew into an infrastructure that ensured information dominance and was a key enabler for American world leadership.

Well into the 1980s, interpretation of photo and satellite images were laboriously done by trained operators.

AI now allows rapid processing and interpretation of these images to enable near real time “actionable intelligence.”

The same technologies are now widely available commercially and are inherently dual use.

Allied leadership in intelligence is now being challenged by the rise of new, open source intelligence systems that have the traditional advantage of open vs. closed systems:   rapid dissemination of technology and knowhow, triangulation and bracketing, or falsification aided by a large distributed pool of experts, novel new applications and use of non-traditional technologies and methods, etc.

There have always been those who are so good at exploiting public, open source information, like the legendary Mark Hibbs, the former editor of the newsletter Nuclearonics Week, who routinely outdid the Intelligence Community (IC), or should we say, Government Intelligence bureaucracy (GIB).

What changed in the 21st Century is developments have enabled the wholesale creation of Mark Hibbs like capabilities operating in the open source domain.

The absolute advantage of the Intelligence Community in high tech method and means, resources, and traditional tradecraft is more and more being matched, and occasionally overmatched by modest open source operations.

Simulation and modeling technologies in the hands of skilled operators enable interpretation of public information in a manner that formerly was the preserve of a small core of government experts.

For example, the Fukushima Daiichi reactor meltdown was recognized by nuclear scientists who tracked public information on the radionuclides and radiation releases, plugged it into their simulator, and concluded there was a meltdown well before TEPCO announced their findings.

Readily available commercial simulators for finite element analysis can build simulation models of rockets, and by plugging in publically available photos or commercial satellite images, obtain solid estimates of the performance parameters of DPRK missile launches comparable to what US surveillance satellite images could do in the 1970s.

The GIB have on more than a few occasions, been soundly beaten by the open-source community.

Most recently, the analysis of North Korea’s intentions and motivations for their WMD programs have been stymied by the inability to depart from the longstanding IC consensus that it is a traditional defensive deterrent for regime survival.

That interpretation fitted nicely into the cold war era model and perception of nuclear weapons as WMDs that cannot be used without causing a civilization or world ending calamity.

Data points about PRC’s nuclear posture and the DPRK nuclear program’s rapid progress since the rise of Kim Jong Un, sharply diverged from the dominant model’s predictions.

IC terminology remained value and assumption laden: calling PRC and DPRK’s nuclear arsenal “deterrents” when it could have been termed “arsenal” that left open the question of intention and motives.

The IC, trapped in their cold war era world view, was slow to change their models in the face of conflicting data.

PRC’s objection to THAAD in ROK is consistent with their need to preserve an offensive tactical nuclear first strike strategy against allied bases in the NE Asia region — a dramatic departure from the PRC “party line” on their nuclear “deterrent”.

Similarly, North Korea’s rapid development of a nuclear arsenal of the breadth, depth, and intercontinental capabilities is far in excess of any imaginable need for regime survival or to deter an attack on DPRK.

The idea that North Korea under Kim Jong Un is motivated to win an offensive war and complete the task of expulsion of the US from the Korean peninsula, unify Koreas on their terms, and seek trillions of “compensation” from their enemies was revealed by old fashioned desk research of open source material.

Analysis of this material with a historical perspective is all it takes to recognize a different dynamic than traditional nuclear deterrence theories assumed.   If one came in with an open mind.

The proposition that North Korea is in it “for the money”, or war for profit, have been arbitrarily dismissed and disregarded by the professional IC until recently even though it was patently clear from open source analysis of their behavioral patterns since 1992.

DPRK’s Foreign Minister Ri’s speech to the UN in September 2017 that explicitly state this goalextortion – was, until recently, dismissed by many “experts”.

A profit minded North Korea is unlikely to be deterred nor are allied dominance in nuclear weapons necessarily usable for this kind of threat.

Failure to predict DPRK’s rapid rise as a nuclear armed state and their motivations and intentions is a clarion call to the IC that radical changes are required to modernize, vastly expand the capabilities and skills of the establishment.

The old “killer apps” that allowed Allied IC to dominate are still useful, but have seen their relative advantages decline.

Searching for new “killer apps” and bringing them to market quickly need to be as high a priority as the Corona program.

The history of Corona inform us that its success was the product of a small group of top officials under President Eisenhower whose steely determination, aggressive risk taking, and stomach for failure after failure was essential.

12 successive launch failures was the price before success.

One wonder whether the risk adverse GIB today have the mojo to take such risks today without a major culture and organization shakeup.   Who will be this generation’s Billy Mitchell (who pioneered Air power), or William Donovan (founded OSS), or Andrew Marshall (ONA)?

The Trump Administration have so far updated the National Security Strategy, National Defense Strategy, and is about to release a new Nuclear Strategy.   Updating the National Intelligence Strategy to align with these new perspectives, and undertaking the transformation of the GIB to restore America’s edge is an urgent priority.

George Tenant recalled fondly how Corona’s founders and pioneers: “They dreamed the impossible. They dared the impossible. And they did the impossible—day in and day out.”

That is what is needed.

Make the impossible possible.

This article was first published on February 9, 2018.