- Run by
- Webz.io
- Type
- Data crawler
- Visits
- On its own schedule
- robots.txt
- Follows robots.txt
- robots.txt name
Webzio- Documentation
- Webz.io's page
What Webzio does
Webzio is Webz.io's web crawler. It collects content from sites and enriches it, with categories and sentiment, for the web data Webz.io sells. A companion, Webzio-extended, tags what it collects as usable or not for AI training, and Webz.io says it only supplies data tagged as permitted for AI use.
Does Webzio follow robots.txt?
Yes. Webz.io says its two bots, Webzio and Webzio-extended, follow robots.txt exclusions.
How to block or allow Webzio
Block it from the whole site:
User-agent: Webzio
Disallow: /Keep it out of one folder only:
User-agent: Webzio
Disallow: /private/ A crawler that finds a group with its own name follows only that group and ignores User-agent: *. If your * group disallows paths such as /admin/, repeat those lines in the Webzio group. robots.txt works per host, so add the rules to each subdomain you want covered.
Test the result with the robots.txt checker, or write a new file with the robots.txt generator.
What blocking Webzio changes
Webz.io stops collecting your pages, so they drop out of the data feeds it sells.
Where to see Webzio visits
Most bots never appear in Google Analytics. Many don't run JavaScript, so the analytics tag never loads, and Google Analytics 4 automatically excludes traffic from known bots and spiders. Your server or CDN logs record every request with its user agent, so search them for the bot's name.
OneLence recognizes Webzio and keeps it out of your visitor counts. With Premium, bot traffic shows which bots visit your site, how often and what kind they are.
Other bots from Webz.io
- Omgilibot: Omgilibot is Webz.io's original crawler, built for the old Omgili search engine. Webz.io's newer Webzio bot builds on it and replaces it.
Frequently asked questions
What is Webzio?
Webzio is Webz.io's web crawler. It collects content from sites and enriches it, with categories and sentiment, for the web data Webz.io sells. A companion, Webzio-extended, tags what it collects as usable or not for AI training, and Webz.io says it only supplies data tagged as permitted for AI use.
Does Webzio follow robots.txt?
Yes. Webz.io says its two bots, Webzio and Webzio-extended, follow robots.txt exclusions.
How do I block Webzio?
Add a group to robots.txt with User-agent: Webzio, then Disallow: /.
What happens if I block Webzio?
Webz.io stops collecting your pages, so they drop out of the data feeds it sells.
Why doesn't Webzio show up in Google Analytics?
Most bots never appear in Google Analytics. Many don't run JavaScript, so the analytics tag never loads, and Google Analytics 4 automatically excludes traffic from known bots and spiders. Your server or CDN logs record every request with its user agent, so search them for the bot's name.