Comments on: CIA Invests in Social Media Monitoring http://blog.operator-speaking.com/2009/10/23/central-intelligence-agency-reads-your-tweets/ Thu, 05 Aug 2010 02:22:51 +0000 hourly 1 http://wordpress.org/?v=3.0.1 By: The Operator http://blog.operator-speaking.com/2009/10/23/central-intelligence-agency-reads-your-tweets/comment-page-1/#comment-851 The Operator Sun, 25 Oct 2009 09:09:08 +0000 http://blog.operator-speaking.com/?p=3953#comment-851 Ah, <a href="http://www.linkedin.com/pub/kevin-burton/0/a/288" rel="nofollow">Mr. Burton</a>, how kind of you to join the discussion. Yes, by all appearances, Spinn3r plays fair - but, all kidding aside, the burden of sorting and validating the data falls upon the data collector and yours is but one of many spiders: I didn't see corresponding robots.txt hits for more than half of the hits to my feed (discounting the direct hits from residential ISP ASN's). It was your spider which left me at the conclusion that I would need to run an ASN check and start using CIDR notation to improve poisoned-feed uptake, so thank you for that. I'm working on the WordPress plugin right now, (not that I expect many will be paranoid or geeky enough to install it) though I would suspect that you've already developed a decent system for filtering out nonsensical spam - just a heads-up. Off-topic: Your possessive pronouns are feeling left out. HAIL ERIS! Ah, Mr. Burton, how kind of you to join the discussion.

Yes, by all appearances, Spinn3r plays fair – but, all kidding aside, the burden of sorting and validating the data falls upon the data collector and yours is but one of many spiders: I didn’t see corresponding robots.txt hits for more than half of the hits to my feed (discounting the direct hits from residential ISP ASN’s).

It was your spider which left me at the conclusion that I would need to run an ASN check and start using CIDR notation to improve poisoned-feed uptake, so thank you for that. I’m working on the WordPress plugin right now, (not that I expect many will be paranoid or geeky enough to install it) though I would suspect that you’ve already developed a decent system for filtering out nonsensical spam – just a heads-up.

Off-topic: Your possessive pronouns are feeling left out. HAIL ERIS!

]]> By: Kevin Burton http://blog.operator-speaking.com/2009/10/23/central-intelligence-agency-reads-your-tweets/comment-page-1/#comment-850 Kevin Burton Sun, 25 Oct 2009 05:56:23 +0000 http://blog.operator-speaking.com/?p=3953#comment-850 Guess how I found this post? :) All kidding aside. You can usually just contact these guys, put up a robots.txt, or just plain IP block the crawlers. If you don't want us to access you're content we're more than happy to comply. Kevin Guess how I found this post? :)

All kidding aside. You can usually just contact these guys, put up a robots.txt, or just plain IP block the crawlers.

If you don’t want us to access you’re content we’re more than happy to comply.

Kevin

]]>