Data Mining

DATA MINING….Here’s a tidbit from the Norristown Times Herald about the nature of the Able Danger project. As usual, the source is a “former defense intelligence official familiar with the program”:

In their efforts to locate terrorists, the operation’s technology analysts used data mining and fusion techniques to search terabyte-sized data sets from open source material ? such as travel manifests, bank transactions, hotel records, credit applications ? and compared this material with classified information.

If this is accurate, it means Able Danger wasn’t exactly the “open source” project we’ve been led to believe. At the very least, it used classified information along with stuff from commercial databases.

But is it accurate? Contrary to conventional wisdom, commercial databases like these are pretty tightly held, and getting complete, free wheeling access to them ? which is the kind you need for genuine data mining ? is pretty unlikely. What’s more, it’s especially unlikely for an experimental project with a small staff, minimal funding, and only a year or two of experience.

This whole thing still doesn’t sound right to me. But I’m passing along the info for the record.

Support Nonprofit Journalism

If you enjoyed this article, consider making a donation to help us produce more like it. The Washington Monthly was founded in 1969 to tell the stories of how government really works—and how to make it work better. Fifty years later, the need for incisive analysis and new, progressive policy ideas is clearer than ever. As a nonprofit, we rely on support from readers like you.

Yes, I’ll make a donation