Operator Speaking by Zachary Constantine

CIA Invests in Social Media Monitoring

2009-10-23 01:23:29 // The Operator

America’s spy agencies want to read your blog posts, keep track of your Twitter updates — even check out your book reviews on Amazon.

In-Q-Tel, the investment arm of the CIA and the wider intelligence community, is putting cash into Visible Technologies, a software firm that specializes in monitoring social media. It’s part of a larger movement within the spy services to get better at using ”open source intelligence” — information that’s publicly available, but often hidden in the flood of TV shows, newspaper articles, blog posts, online videos and radio reports generated every day.

. . .

“Anything that is out in the open is fair game for collection,” says Steven Aftergood, who tracks intelligence issues at the Federation of American Scientists. But “even if information is openly gathered by intelligence agencies it would still be problematic if it were used for unauthorized domestic investigations or operations. Intelligence agencies or employees might be tempted to use the tools at their disposal to compile information on political figures, critics, journalists or others, and to exploit such information for political advantage. That is not permissible even if all of the information in question is technically ‘open source.’”

- U.S. Spies Buy Stake in Firm That Monitors Blogs, Tweets
by Noah Shachtman for Wired.com

via AdScam

That almost explains that hit why the CIA is wasting time at my site… perhaps they’re genuinely interested in what private citizens have to say?

More likely, it’s about finding what information has leaked.

Should this be a concern to everyday bloggers? For the same reason it’s a bad idea to have your fingerprint in a government database (a false match rate of 0.01177% means that there is a high probability of a false match against your print if you’re in a pool of hundreds of millions) it is a bad idea to allow your information to be aggregated and monitored by government agencies without your explicit permission – the best thing that can come from spies reading your blog is nothing (and the worst thing is … well, let’s skip the tin foil hat conjecture and just assume that if nothing good will come of it, there’s no point in allowing it).

How do I protect my Twitter/Facebook/MySpace account?

You can’t – and that is why you should not share your personal information with profit-driven corporations.

For the same reason you don’t give out your name and address to strangers (particularly the ones with clipboards) on the street, you should maintain some level of access policy to the personal information you choose to publish on the internet – and no, the “privacy controls” so graciously granted you by the aforementioned profit-driven corporations don’t mean a thing when there’s a dollar to be made (think business-to-business information resale).

How do I protect my information on sites I control?

I have a solution in mind, however, I will need quite a bit of time to make the solution viable (and there remain some usability issues which would need to be addressed) – the short answer would be to block the IP addresses of offending spiders.

Unfortunately, they are legion.

I checked the blog.operator-speaking.com logs and found (much to my dismay) that rat-bastards like Gavin Gibbons are using services like URLFan and Radian6 to “keep tabs” on sites like my own: (and possibly yours, no?)

Radian6 is a social media monitoring company that has developed a product offering specifically tailored to PR and marketing companies, that helps companies find and listen to conversations about their brands.

- Radian6 Crawler

Access large amounts of breaking news and other business information handcrafted to your exacting specifications. Go get it or we’ll deliver it to you. Use it to inform, innovate and investigate your world, from the competitor down the block to trends across the globe. Every day, you can track and analyze competitor movement, follow customer news.

- moreover technologies

Spinn3r is indexing your site on behalf of our user base to provide your content so it can influence their applications. We’re used by search engines, analytic services, competitive intelligence services, etc.

- spinn3r

… and those are just “visitors” willing to reveal their identities. I found a total of seven feed-search bots and four unknowns (three of which had IP addresses issued by commercial datacenters) – block IP addresses at your discretion; I plan to offer these parasites something nastier to chew on.

Update 10/24/2009:

See This Means War for an example of the crap-tastic content feed search spiders are eating right now at the Operator Speaking buffet.

Update 10/26/2009:

Aww hell, just go to feed_er.php and take a look for yourself. Comedy goldmine.

All the automatic tellers with their cameras acting as eyes, they ingest, digest, and regurgitate paper… and watch for the database – maybe those lines lead back to Ohio, maybe they run back down to the murky floor of the harbor where words like “Novus Ordo Seclorum” are coined

- Booze, Cthulhu, and the Weeds
by Z. Constantine
[Spoken word excerpt]

3 Responses to “CIA Invests in Social Media Monitoring”

  1. Guess how I found this post? :)

    All kidding aside. You can usually just contact these guys, put up a robots.txt, or just plain IP block the crawlers.

    If you don’t want us to access you’re content we’re more than happy to comply.


  2. Ah, Mr. Burton, how kind of you to join the discussion.

    Yes, by all appearances, Spinn3r plays fair – but, all kidding aside, the burden of sorting and validating the data falls upon the data collector and yours is but one of many spiders: I didn’t see corresponding robots.txt hits for more than half of the hits to my feed (discounting the direct hits from residential ISP ASN’s).

    It was your spider which left me at the conclusion that I would need to run an ASN check and start using CIDR notation to improve poisoned-feed uptake, so thank you for that. I’m working on the WordPress plugin right now, (not that I expect many will be paranoid or geeky enough to install it) though I would suspect that you’ve already developed a decent system for filtering out nonsensical spam – just a heads-up.

    Off-topic: Your possessive pronouns are feeling left out. HAIL ERIS!

  3. me Says:

    It’s scary. It reminds me of the book 1984 (which I have almost finished reading).
    On the one hand the internet is great because it allows us to talk to people all over the world and spread their opinions but of course, now governments have caught on and scared that they wont be able to control peoples opinions as easily.

    I read an article a while ago saying that the UK government is actively trying to infiltrate conspiracy theory chat rooms etc in order to counteract peoples arguments.
    I’m not saying all conspiracy theories are true but if people are unable to discuss whatever they like without being secretly manipulated then it is yet another blow to free speech.
    Anyway, nice article.

Leave a Reply