Sitemaps and DoS

[Azareal]

Implementing sitemaps makes me kind of nervous without really strict ratelimiting.
It seems like it might one of the most DoS-able things on the face of the planet (well, not literally lol).

Ridiculously large files which can be requested by n number of bots from m IP addresses at any time they want. Even a misconfigured bot could probably cause havoc, hm.

 

[TopSilver]

Not sure if sitemaps would ever cause that probably since they are written in XML form. Though now Xenforos sitemaps have became .php extended. Either way I don't think that would be much of an issue for a Dos attack. Usually DDoS or DoS attacks happen because a user has got your ip address from a MX entry from emails. This tends to cause issue for a lot of people. With Cloudflare now having free advanced DDoS protection I can see that it would only anger the attacker haha as they would not get very far

 

[Azareal]

Not sure if sitemaps would ever cause that probably since they are written in XML form. Though now Xenforos sitemaps have became .php extended. Either way I don't think that would be much of an issue for a Dos attack. Usually DDoS or DoS attacks happen because a user has got your ip address from a MX entry from emails. This tends to cause issue for a lot of people. With Cloudflare now having free advanced DDoS protection I can see that it would only anger the attacker haha as they would not get very far

The spec allows sitemaps, particularly sitemap indices, to be as large as 20MB.
They are also not that easy to break up. Somewhere in the millions of posts, it might get a little nasty.

Also, a single endpoint may load as many as ten thousand items, even in non-exceptional cases. Maybe, it could be cached in memory rather than relying on the filesystem or the database's algorithms for smaller cases, although requests for it shouldn't be common enough to really need that extra resource use.

It's been a while since I did research into it though.