How do bots work?

[TheExoticFish.com]

Wow that's crazy.

Yahoo Slurp has visited my forum 1101 times this month and used 5.16 MB of bandwidth.

Do you know off hand how much they were using on here ?

 

[CyberFreak]

We blocked it in June and this is bandwidt used with Gzip enabled on a light skin bandwidth-wise. Google bot is always our second most active bot after Yahoo.

May

Bot--------------Hits--------BandWidth--Last visit
Yahoo Slurp------61083+851-603.82 MB--31 May 2008 - 23:59
Googlebot--------23030+55--165.65 MB--31 May 2008 - 23:49

April

Yahoo Slurp------68845+2504-439.65 MB-30 Apr 2008 - 23:57
Googlebot--------5867+51-----33.78 MB--30 Apr 2008 - 23:39

 

[TheExoticFish.com]

Woah! That's a serious problem. Hopefully that doesn't happen to my forum.

 

[abefroman]

I'd move my forum to a dedicated server before I'd block the bots.

Do you want SE listings?

 

[CyberFreak]

We used to get crap traffic from yahoo anyway. We get way more from google. Plus we don't have the money just to upgrade to keep the yahoo bot from overloading us.