BotBouncerBotBouncer / AI bots / Omgilibot
Omgilibot
Webz.io, a training and dataset crawler
- Operator
- Webz.io
- Type
- Model training and datasets
- Sent as a user agent
- Not confirmed
- Follows robots.txt
- Not confirmed
- Details
- Not yet checked against the operator's own documentation
What it does
Webz.io crawler that collects forum and news data it sells. Webz.io may have renamed it.
We have not yet checked this page against Webz.io's own documentation. Check it before you rely on it.
Block it in robots.txt
User-agent: Omgilibot Disallow: /
To allow it again, remove those lines or use Allow: /. Rules for other bots are not affected.
Block it on your server
robots.txt is a request. To enforce the block, filter the user agent on your server.
Nginx
map $http_user_agent $blocked_bot {
default 0;
~*(Omgilibot) 1;
}
# inside server { }
if ($blocked_bot) { return 403; }Apache (.htaccess)
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Omgilibot [NC]
RewriteRule ^ - [F,L]Cloudflare custom rule (action: Block)
(lower(http.user_agent) contains "omgilibot")
Source
No official documentation link has been confirmed yet.