Magazine article National Defense

U.S. Government Turns to Crowdsourcing for Intelligence

Magazine article National Defense

U.S. Government Turns to Crowdsourcing for Intelligence

Article excerpt

* The Pentagon and U.S. intelligence community spend billions of dollars each year trying, with mild success at best, to predict the future.

They organize elaborate wargames, develop computer algorithms to digest information and rely on old-fashioned aggregation of professional opinion.

Past intelligence failures have been costly and damaging to U.S. national security. Trying to avoid previous pitfalls, agencies are on a constant treasure hunt for new technologies that might give them an edge.

The Intelligence Advanced Research Projects Activity in February solicited industry proposals for how to improve the accuracy of intelligence forecasting. Under the auspices of the Office of the Director of National Intelligence, IARPA invests in research programs that provide an "overwhelming intelligence advantage over future adversaries."

Applied Research Associates, a New Mexico-based firm, has launched a program it hopes will improve upon the traditional methods of gathering expert opinion by using computer software that could make better-informed predictions. The system chooses the best sources of information from a huge pool of participants.

ARA won the bid and started working on its Aggregative Contingent Estimation System, or ACES, in May.

The firm's southeast division, headquartered in Raleigh, N.G, has teamed with seven universities to devise a method of farming out global intelligence questions to the general public through the Internet. It began collecting crowdsourced opinions in early July.

Crowdsourcing is a method of problem solving where a task is doled out to an undefined group of people through an open call to participate.

"Anyone can sign up; the more the merrier," said Dirk Warnaar; ARAs principal investigator for the ACES project. Participants interested in a range of topics, including politics, military, economics, science, technology and social affairs are invited to register at:

"You can look at the crowd as people who are on the ground in real-life situations who have the best information," Warnaar said. "Think of it like a large group of foot soldiers providing feedback."


The crowd should eventually be able to provide more accurate predictions on global conflict in a time of increased uncertainty Warnaar said.

"We don't want to rule anyone out," Warnaar said in an interview. "The term 'expert' is very poorly defined. Some research has shown that some experts are so close to the subject matter at hand that they can form biases and may not be good forecasters."

Aggregated results are being tested over four years with ACES predictions being constantly measured against real-world events, Warnaar said.

The process is not limited to national-security or global conflict; questions of social and economic futures are also being posed to the 1,800 or so participants currently signed up to offer insight. So far about 50 questions have gone through the process, with 14 having resulted in predictions that will be evaluated and measured by IARPA.

But the capability of the program is limited to what questions are asked, Warnaar said. The system is not well suited for such events as predicting the earthquake that hit Japan in March, causing a tsunami and subsequent nuclear meltdown.

"In a case like that, we probably wouldn't have thought to ask whether an earthquake would hit Japan. But there are still a lot of questions out there we could ask based on current events and extensions of those" that could impact national security.

The goal is to demonstrate better accuracy in predicting near-term and middle-term events than an opinion poll by the end of the four-year experiment. In the first year, Warnaar is seeking to achieve a 20 percent improvement over traditional polling methods. If its predictions turn out more accurate, the program will be made available to government decision makers. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.