Aggressive MSN Search Bots

[Dangerousdave26 1]

Tom

 

I thought I might share this tidbit Ickie came up with over at the Outhouse.

 

http://www.sim-outhouse.com/sohforums/showthread.php?79790-Aggressive-MSN-Search-Bots

 

I noticed AVSIM was down the other day and it looks like the site is loading slow (for me).

 

I am thinking you might be having the same issue with the MSN Search Bots almost acting like a DDoS Attack.

 

http://social.microsoft.com/Forums/en-US/d4743e76-d7e7-49d2-84a5-3fd119dd6438/msn-bot-crashes-my-site-like-a-dos-attack

 

http://social.microsoft.com/Forums/en-US/41092b13-7fd6-4c65-a43d-fe81add8e964/msn-search-bot-trying-to-attack-my-site

 

http://social.microsoft.com/Forums/en-US/41092b13-7fd6-4c65-a43d-fe81add8e964/msn-search-bot-trying-to-attack-my-site

 

Ickie has a fix for the issue posted in our thread above.

 

Even if you are not suffering from the above issues with MSN search bots you may want to apply the fix to your server.

 

Dave

 

[Chase 303]

Dave, thank you for sharing this information with us. I'll make sure Tom gets this if he hasn't already.

[Tom Allensworth 1,532]

 

 

Even if you are not suffering from the above issues with MSN search bots you may want to apply the fix to your server.

 

Thank you Dave. There are a number of those that we have taken care of. However, he might get better results by putting those in an .htaccess file rather than the robots.txt file. The .htaccess file block access by the denied IP to ALL areas / directories under the .htaccess root directory rather than the robot.txt file which "bad" robots don't pay any attention to anyway. MSN at this time could be considered a "bad" robot and there are many others that just blow by the instructions in the robot file.

[Dangerousdave26 1]

Originally we tried the Robot.txt file but they just ignored it.

 

We ended just blocking them at the server level in the Firewall.

 

 

[Tom Allensworth 1,532]

Originally we tried the Robot.txt file but they just ignored it. We ended just blocking them at the server level in the Firewall.

 

That will do the job.