How do bots work?

When you say "Bots" I immediately think about the bots like Google and stuff. The most common one is Google and what they do it bookmark your forum pages on there search engines. However, make sure you enable bots as "Guests" because links like access to the admin CP or members profile could be posted all over Google.
 
Yeah, they index posts on your forum in there search engines.

They are the most active members here -lol-
 
Search bot like the bots that show up on this forum aren't bad and just index all the pages and topics and that means when you search for something that is mentioned on your forum it will turn up in the results.

There are some spam bots that are programmed to sign up for forums and spam them but they don't show up on forum under the bot group... manly because they are hard to detect as they have millions of different IPs and useragents.
 
I wonder if any forums have had all 50 bots come on at some point

(probably the main phpBB forum)
 
I had someone PM me the other day telling me I needed to block the bots because they can "let virus' and worms in". LoL
 
Ignore them... Search bots like yahoo,msn and all the others that phpBB recognises won't cause damage... There are some bots that may do this but there is no way of blocking them and don't worry about them.
 
Thinking about it it is possible to spoof (pretend) to be a search bot and actually be a dangerous bot disguised as a search bot but again there is no way of stopping them either really.
 
By default phpBB3 detects bots by their useragents so people can make a bot and set there useragent as googles for example and they will be treated as that bot. If you add a custom bot using IP's then they can't unless they have access to an IP in that range or if they fake an IP.
 
I've edited all my bots and placed specific IPs for them.

Yeah, but faking an IP isn't an easy task. -😉-

Plus making sure they have very limited access should help.
 
Some search engines have various different ranges... Yahoo is a big example of that so to pick up every Yahoo bot you would need to set a huge range of IP's that would pick up other IP's not related to Yahoo.
 
I noticed when i tried to block the bloody thing... We were being bombarded by it through various different IP's constantly and when i block each range it would come back with another.

The yahoo bot is a pain to say the least.
 
We had a crawl limit to stop bots hitting us hard and where as google send one bot on at a time and it follow that limit. Yahoo decided to be idiots over it and have each bot follow the crawl limit but then send about 10 different yahoo bots on at once with different ip's with each following the crawl limit although it was pointless having the limit if they keep sending new bots on.

I will never forget that time i noticed what they were up to while checking the who's online page and i noticed their ip was constantly changing everytime i refreshed it which was every couple of seconds at some points and then i thought screw this and blocked them via the robots.txt file although after about 2 or 3 days they were still doing it and after previous problems with hosts about cpu usage and the fact that we got hardly any hits from yahoo i thought i may aswell ban their IP's as they weren't following the robots.txt file that they download every 24 hours.
 
You must log in or register to reply here.

About Us

Since 2007, Forum Promotion has specialized in providing advertising solutions to webmasters looking to promote their communities. We pride ourselves in being the bridge that connects forum administrators, bloggers, and more.

Sponsors

Affiliates

    Universal Gaming The Coffee House

Stay Connected

Back
Top Bottom