Jump to content
Infinite Unreality Forums
Sign in to follow this  
KaizenNeko

Goodbye all you creepy-crawlies!

Recommended Posts

I had previously blocked GoogleBot from crawling the board because it started to make multiple connections at a time.

 

Since doing so, the others (more MSN & Yahoo) and ramped up their crawling. I didn't intend for it be a spider free-for-all and will not tolerate it any longer. Effective immediately, ALL search engine spiders (actually any robot that uses the robots.txt protocol) has been blocked.

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.
Sign in to follow this  

×