Important bots for a young phpbb site

[Jaosals]

Removed.

 

[CyberFreak]

You can't choose what bots visit your site... They just turn up.

 

[Jaosals]

Removed.

 

[Subz]

You should enable all the bot o visit

 

[Tecca]

Yeah, enable all of them. All are important, though Google is definitely the one you'll almost always be seeing (and should be wanting there).

 

[CyberFreak]

Only because you enable them that doesn't mean they will visit. If you disable one that would stop them visiting.

 

[Dylan]

You could always just put it in robots.txt.Check Forum Promotions robots.txt file forumpromotion.net/robots.txt

 

[Ascent]

Google and Yahoo are the only ones you'll need. All other search engines are almost all ineffective for your site.

 

[Jaosals]

Removed.

 

[Victim]

You should check out this topic, it's about Twiceler bot, it's been causing a lot of problems for web sites.

 

[CyberFreak]

We used to have a problem with the Twiceler bot but adding a Crawl delay in the Robots Txt file sorted it out. The Yahoo bot used to cause way more problems with the number of pages that used to request.