4 Oct
2021
4 Oct
'21
13:37
On Monday 2021-10-04 14:00, Henne Vogelsang wrote:
We had multiple spiders (bing, yandex, unicom, sprinthost.ru) crawling 'expensive' routes like
/package/view_file/...
Every once in a while their frequencies align and then it becomes a bit too much. But in general we want them to index things of course.
Except perhaps home:?