AdsBot-Google: what it is and how to block or allow it

What AdsBot-Google does

AdsBot-Google is Google Ads' crawler. It visits ad landing pages to check their quality, and AdsBot-Google-Mobile does the same for mobile pages. Its visits come from your own ads, or from Dynamic Search Ads targets, which Google lists among the causes of unexpectedly heavy crawling.

• A Disallow under User-agent: * doesn't stop AdsBot. Name AdsBot-Google and AdsBot-Google-Mobile in their own group. Source
• Google uses AdsBot-Google for ads and Googlebot for organic results, and allowing one doesn't allow the other. Source

Does AdsBot-Google follow robots.txt?

Partly. Google classes AdsBot as a special-case crawler: it obeys rules written for AdsBot-Google, but ignores the User-agent: * group, which Google says is done with the advertiser's permission.

It doesn't honor Crawl-delay.

How to block or allow AdsBot-Google

Block it from the whole site:

User-agent: AdsBot-Google
User-agent: AdsBot-Google-Mobile
Disallow: /

Keep it out of one folder only:

User-agent: AdsBot-Google
User-agent: AdsBot-Google-Mobile
Disallow: /private/

A crawler that finds a group with its own name follows only that group and ignores User-agent: *. If your * group disallows paths such as /admin/, repeat those lines in the AdsBot-Google group. robots.txt works per host, so add the rules to each subdomain you want covered.

Because AdsBot-Google doesn't always obey robots.txt, a firewall or CDN rule that matches its user agent is the dependable way to stop it.

Test the result with the robots.txt checker, or write a new file with the robots.txt generator.

What blocking AdsBot-Google changes

Google Ads can't check your landing pages, and ads pointing to them can be disapproved as "Destination not crawlable". Repeated violations can lead to account suspension after at least seven days' warning.

Where to see AdsBot-Google visits

Most bots never appear in Google Analytics. Many don't run JavaScript, so the analytics tag never loads, and Google Analytics 4 automatically excludes traffic from known bots and spiders. Your server or CDN logs record every request with its user agent, so search them for the bot's name.

OneLence recognizes AdsBot-Google and keeps it out of your visitor counts. With Premium, bot traffic shows which bots visit your site, how often and what kind they are.

Other bots from Google

• Google-Extended: Not a crawler: the robots.txt setting for Gemini training and grounding. It doesn't affect Google Search.
• Googlebot: Googlebot is Google Search's crawler. Its crawl also decides whether your pages can appear in AI Overviews and AI Mode, so blocking it takes you out of both.
• Google-InspectionTool: Google-InspectionTool fetches your page when someone runs one of Google's Search testing tools, such as the Rich Results Test or URL Inspection in Search Console.
• Storebot-Google: Storebot-Google is Google's shopping crawler. It fetches pages for Google Shopping surfaces such as the Shopping tab in Search.
• GoogleOther: GoogleOther is a general-purpose Google crawler that product teams use for jobs outside Search, such as research crawls. Blocking it doesn't affect Google Search.
• Google-Agent: Google-Agent is the name AI agents running on Google's infrastructure use when they browse sites and take actions because a user asked them to.
• Google-GeminiNotebook: Google-GeminiNotebook fetches a page when a Gemini Notebook user adds it as a source. It replaced the NotebookLM fetcher in July 2026.
• Google-Read-Aloud: Google-Read-Aloud fetches a page when someone asks Google to read it out loud. robots.txt doesn't stop it; a meta tag does.
• Mediapartners-Google: Mediapartners-Google is the AdSense crawler. It reads pages that carry Google ad tags to pick relevant ads, and it ignores the User-agent: * group.