- Run by
- DuckDuckGo
- Type
- Search engine crawler
- Visits
- On its own schedule
- robots.txt
- Follows robots.txt
- robots.txt name
DuckDuckBot- Documentation
- DuckDuckGo's page
What DuckDuckBot does
DuckDuckBot feeds DuckDuckGo's own search indexes. DuckDuckGo says most of the standard links and images in its results come largely from Bing rather than from its own crawler.
- To remove a standard link from DuckDuckGo, you file the request with Bing, and the change flows through. Source
- DuckDuckGo's AI-assisted answers use a separate fetcher, DuckAssistBot, with its own robots.txt name. Source
Does DuckDuckBot follow robots.txt?
Yes. DuckDuckGo says DuckDuckBot respects WWW::RobotRules, a standard library for reading robots.txt rules.
How to block or allow DuckDuckBot
Block it from the whole site:
User-agent: DuckDuckBot
Disallow: /Keep it out of one folder only:
User-agent: DuckDuckBot
Disallow: /private/ A crawler that finds a group with its own name follows only that group and ignores User-agent: *. If your * group disallows paths such as /admin/, repeat those lines in the DuckDuckBot group. robots.txt works per host, so add the rules to each subdomain you want covered.
Test the result with the robots.txt checker, or write a new file with the robots.txt generator.
What blocking DuckDuckBot changes
DuckDuckGo's own crawler stops fetching your pages, but you can still appear in its results through Bing.
Where to see DuckDuckBot visits
Most bots never appear in Google Analytics. Many don't run JavaScript, so the analytics tag never loads, and Google Analytics 4 automatically excludes traffic from known bots and spiders. Your server or CDN logs record every request with its user agent, so search them for the bot's name.
OneLence recognizes DuckDuckBot and keeps it out of your visitor counts. With Premium, bot traffic shows which bots visit your site, how often and what kind they are.
Other bots from DuckDuckGo
- DuckAssistBot: DuckAssistBot fetches pages in real time for DuckDuckGo's AI-assisted answers. Blocking it takes you out of those answers without touching your search rankings.
Frequently asked questions
What is DuckDuckBot?
DuckDuckBot feeds DuckDuckGo's own search indexes. DuckDuckGo says most of the standard links and images in its results come largely from Bing rather than from its own crawler.
Does DuckDuckBot follow robots.txt?
Yes. DuckDuckGo says DuckDuckBot respects WWW::RobotRules, a standard library for reading robots.txt rules.
How do I block DuckDuckBot?
Add a group to robots.txt with User-agent: DuckDuckBot, then Disallow: /.
What happens if I block DuckDuckBot?
DuckDuckGo's own crawler stops fetching your pages, but you can still appear in its results through Bing.
Why doesn't DuckDuckBot show up in Google Analytics?
Most bots never appear in Google Analytics. Many don't run JavaScript, so the analytics tag never loads, and Google Analytics 4 automatically excludes traffic from known bots and spiders. Your server or CDN logs record every request with its user agent, so search them for the bot's name.