From late 2004 until mid-2006, a little-known data-mining
computer system developed by the US Department of Homeland Security
to hunt terrorists, weapons of mass destruction, and biological
weapons sifted through Americans' personal data with little regard
for federal privacy laws.
Now the $42 million cutting-edge system, designed to process
trillions of pieces of data, has been halted and could be canceled
pending data-privacy reviews, according to a newly released report
to Congress by the DHS's own internal watchdog.
Data mining to help fight the war on terror has become an
accepted, even mandated, method to provide timely security
information. The DHS operates at least a dozen such programs;
intelligence agencies and the Department of Defense employ many
But ADVISE (AnalyA-sis, DisseminaA-tion, VisuA-aliA-zation,
Insight and Semantic EnhanceA-ment) was special. An electronic
omnivore conceived in 2003, it was designed to ingest information
from scores of databases, blogs, e-mail traffic, intelligence
reports, and other sources, government documents and researchers
Sifting that enormous mass at lightning speed, ADVISE was to
display data patterns visually as "semantic graphs" - a sort of
illuminated information constellation - in which an analyst's eye
could spot links between people, places, events, travel, calls, and
Report: DHS didn't follow guidelines
Yet ADVISE, whose existence and scope were first detailed by the
Monitor in February 2006, seems to have run afoul of its own
ambitious scope. It failed to incorporate federal privacy laws into
its system design. From its earliest days, the system's pilot
programs used "live data, including personally identifiable
information, from multiple sources in attempts to identify potential
terrorist activity," but without taking steps required by federal
law and DHS's own internal guidelines to keep that data from being
misused, the DHS Office of Inspector General (OIG) said in a June
report to Congress, which was made public Aug. 13.
In a rebuttal attached to the report, the DHS Directorate for
Science and Technology disagreed with most of the OIG's findings.
"The ADVISE tool set is little more than an empty framework to which
data must be applied," wrote Jay Cohen, DHS undersecretary for
science and technology, in a letter accompanying the rebuttal. He
said no privacy laws were violated.
Even in searching for terrorists, data-mining programs are
supposed to ensure that Americans' personal information is used only
when necessary and lawful - and only for specific and proper uses.
One problem is that even data that look anonymous aren't necessarily
so. For instance, even when names and Social Security numbers are
stripped from data files, programmers can still identify 87 percent
of Americans through their date of birth, gender, and five-digit Zip
Code, researchers say. So a system has to be carefully designed and
use encryption and other computer techniques to comply with the law.
Last week the Pentagon shut down its TALON terrorism database
program, which had been found to hold files on peace activists. In
2003, another military data-mining project - the Total Information
Awareness project - was also ended following a congressional uproar
over privacy fears.
Congress last fall ordered its Government Accountability Office
to audit the program for privacy and effectiveness. …