NitaqoBot

The fetch user agent is NitaqoBot/1.0 (+https://nitaqo.com/bot). It requests robots.txt and then one homepage, up to 2 MB, with no script execution. It does not download the homepage body when URLhaus, a UT1 malware or phishing list, or a Web Risk MALWARE verdict names the host. Every other name is fetched the same way.

Rates: about one RDAP request per second per registry base URL, homepage fetches burst two and then one every two seconds per host, WHOIS about one query per second per server, crt.sh at most five requests a minute, Cert Spotter well under 100 an hour when a key is set. HTTP 429 responses honor Retry-After.

To opt out of homepage fetches, publish a robots.txt rule for User-agent: NitaqoBot with Disallow: /. DNS and RDAP are registry and resolver lookups, not HTML crawls, and are not controlled by robots.txt.