Twiceler Bot, Causing Problems?

[Victim]

Seems that the Twiceler bot that crawls thousands of sites is causing a lot of problems.

Twiceler is used for the Cuil (pronounced cool) search engine, which they say is the best search engine on the net and has more indexed web pages than any other search engine, and they also say they have three times as many indexed web pages as Google and ten times as many as Microsoft!

Here is a article about some of the problems it has been causing: click here to go to article

Cuil can be accessed from here. It sure looks nice on first impressions, but when searching can kind of be confusing with the pictures and layout of the search results.

Want more information about Cuil? Click here.

 

[Rich Edmonds]

It's just a terribly bad effort to challenge Google from ex Google employees. They could have made it so much better for users before PR and releasing it out on the net, but pumping more money into it isn't going to solve things, they need to sort it out and this isn't helping matters.

 

[Dylan]

It browsed here recently.

 

[CyberFreak]

We used to have a problem with it but the Yahoo bot caused us more of a problem with their stupid crawling method which was meaning Yahoo was loading a page every 2 seconds.

 

[Comkid]

I've never seen this bot around...

I hope they shut down Cuil forever They should fix this quickly before they get sued -evil-

 

[AnAccount]

I don't think I've ever seen this bot around... I'll have to keep an eye out for it.

 

[CyberFreak]

I've never seen this bot around...

I hope they shut down Cuil forever They should fix this quickly before they get sued -evil-

A bit extreme... You can't say anything like this just based on a story on a site. Especially if you haven't had it on your site. If Cuil get sued then so should Yahoo based on my experience with these 2 bots.

 

[Comkid]

Really, Well What has Yahoo done?

Oh, then they should have to pay for all the servers they broke with the bot -evil-

 

[Merlin]

It's not in my bots group... Ya think I could ban it if it comes?

 

[Rich Edmonds]

It's not in my bots group... Ya think I could ban it if it comes?

A simple addition to your robots.txt should tell it to politely go away. As you are on a free host you probably won't be able to create one and you can't add a meta tag either so I guess you would need to add it to the bot group?