- Run by
- Webz.io
- Type
- Data crawler
- Visits
- On its own schedule
- robots.txt
- Follows robots.txt
- robots.txt name
Omgilibot- Documentation
- Webz.io's page
What Omgilibot does
Omgilibot was built for Omgili, a search engine that has since closed, and later collected news, blogs, reviews, shop and forum content for Webz.io's data API.
Does Omgilibot follow robots.txt?
Yes. Webz.io says owners can opt out directly or through robots.txt, and that it always complies.
How to block or allow Omgilibot
Block it from the whole site:
User-agent: Omgilibot
Disallow: /Keep it out of one folder only:
User-agent: Omgilibot
Disallow: /private/ A crawler that finds a group with its own name follows only that group and ignores User-agent: *. If your * group disallows paths such as /admin/, repeat those lines in the Omgilibot group. robots.txt works per host, so add the rules to each subdomain you want covered.
Webz.io also takes opt-out requests directly, and says it will deal with reports of the bot slowing a site.
Test the result with the robots.txt checker, or write a new file with the robots.txt generator.
What blocking Omgilibot changes
Webz.io stops collecting your pages for its web data feeds.
Where to see Omgilibot visits
Most bots never appear in Google Analytics. Many don't run JavaScript, so the analytics tag never loads, and Google Analytics 4 automatically excludes traffic from known bots and spiders. Your server or CDN logs record every request with its user agent, so search them for the bot's name.
OneLence recognizes Omgilibot and keeps it out of your visitor counts. With Premium, bot traffic shows which bots visit your site, how often and what kind they are.
Other bots from Webz.io
- Webzio: Webzio collects and structures web content for the data feeds Webz.io sells to other companies. It replaced the older Omgilibot.
Frequently asked questions
What is Omgilibot?
Omgilibot was built for Omgili, a search engine that has since closed, and later collected news, blogs, reviews, shop and forum content for Webz.io's data API.
Does Omgilibot follow robots.txt?
Yes. Webz.io says owners can opt out directly or through robots.txt, and that it always complies.
How do I block Omgilibot?
Add a group to robots.txt with User-agent: Omgilibot, then Disallow: /. Webz.io also takes opt-out requests directly, and says it will deal with reports of the bot slowing a site.
What happens if I block Omgilibot?
Webz.io stops collecting your pages for its web data feeds.
Why doesn't Omgilibot show up in Google Analytics?
Most bots never appear in Google Analytics. Many don't run JavaScript, so the analytics tag never loads, and Google Analytics 4 automatically excludes traffic from known bots and spiders. Your server or CDN logs record every request with its user agent, so search them for the bot's name.