How many hits per month from a search engine spider is good?

abefroman

Familiar Face
Joined
Jul 23, 2008
Messages
53
Reaction score
0
FP$
6
How many hits per month from a search engine spider (for example the googlebot) is considered good?

How many does your forum get?
 
So far this month

+## equals number of hits on robots.txt file

Googlebot 12989+50
Unknown robot (identified by 'bot/' or 'bot-') 3173+24
MSNBot 170+185
Unknown robot (identified by 'crawl') 214+131
Unknown robot (identified by 'spider') 178+105
Alexa (IA Archiver) 251+25
MSNBot-media 231+42
The web archive (IA Archiver) 0+157
Unknown robot (identified by hit on 'robots.txt') 0+88
Ask 18+22
Yahoo! Slurp China 11+15
Google Sitemaps 6
Unknown robot (identified by 'robot') 3+2
Relevant Noise 2
Walhello appie 0+1


We had banned the yahoo bot for reasons mentioned here - viewtopic.php?f=54&t=8538&start=15
 
Wow thats good. How do I find out what bot is what I only know the IP addresses of my users. I have forumer, and it doesnt tell me if its a bot or what.

I would say theres alot of google bots going around. I get prolly 25+ a day.
 
If it doesn't detect them automatically then the only way is to trace every guests ip although you can't do that 24 hours a day. I got ours from a stats program called AW stats through our cpanel.
 
they're on almost all the time.. dont know how to tell per month though
 
Back
Top Bottom