Open page
This page is allowed for every crawler and exists as the control for the closed pages beside it. Fetches of this page by automated crawlers are recorded with the crawler's declared name and the date. There is nothing else here.
This page is allowed for every crawler and exists as the control for the closed pages beside it. Fetches of this page by automated crawlers are recorded with the crawler's declared name and the date. There is nothing else here.