Facebook prohibits unauthorised crawling

Facebook has made changes to its social networking site in order to stop crawling activity by unauthorised users.

The changes, introduced by modifying the Facebook robot.txt file including crawling data settings, specify that only certain search engines (such as Google and Microsoft's Bing) have authorised access to perform such activities.

Facebook attempted to sue Pete Warden, associate of www.fanpageanalytics.com after he used data crawling on Facebook to gather user information for the website. Following the legal case, Facebook decided to modify their current crawling settings in order to improve privacy and user safety.

They have insisted that they will grant crawling permission to any legitamite search outfits or sources provided that their reasons are appropriate. Written permission will be granted to all successful applications.

A blog post from Facebook CTO Bret Taylor can be found here regarding the subject.