Allow our crawler through your firewall.

Your audit was blocked by a firewall or security service on your site. Allow the User-Agent below and the next scheduled audit runs automatically, nothing to click in the app.

Updated 6 min readWebsite Health

What to allow.

Bot name
LighthouseLocal Bot
User-Agent token to match
LighthouseLocalBot
Full User-Agent
LighthouseLocalBot/1.0 (lighthouse; +https://www.lighthouselocal.ai/bot) (Beacon audits send seo-report in place of lighthouse)
IP addresses
None published. The crawler has no fixed IP range, so allow by User-Agent, not by IP.
Robots.txt token
LighthouseLocalBot
Verification
Every request is signed (Web Bot Auth, RFC 9421). See how to verify a request.

Match on the substring LighthouseLocalBot, so a version bump does not break your rule.

Find what is blocking us.

The block usually comes from one of four places: a CDN or firewall in front of the site (Cloudflare, Sucuri), a WordPress security plugin (Wordfence), your hosting company, or a rule on the web server itself. Site builders (Wix, Squarespace, Shopify, Webflow) do not block unknown crawlers by default; if you are on one of those, check robots.txt (below) first. If you do not know which applies, forward this page to whoever manages your website.

Cloudflare.

  1. Open Security > Security rules (WAF) > Create rule.
  2. Paste the expression:
    (http.user_agent contains "LighthouseLocalBot")
  3. Set the action to Skip. Under the Skip options tick "All remaining custom rules" (a likely cause: your own block-unknown-bots custom rule), "All rate limiting rules", "All Super Bot Fight Mode rules" and "All WAF Managed Rules rules".
  4. Place the rule above your blocking rules.

The "Block AI bots" / AI Crawl Control setting can catch unverified crawlers on behavior. If it is on, add the Skip rule above it or set it to log first. LighthouseLocalBot is not yet on Cloudflare's Verified Bots list, so the User-Agent rule is the way to allow it today.

WordPress security plugins.

Wordfence
It has no allow-by-User-Agent setting. Check Firewall > Blocking for a custom pattern that matches our User-Agent and remove it. Then go to Firewall > Manage WAF > Rate Limiting and raise "If a crawler's page views exceed" (our audit fetches at most about 16 pages, 4 at a time), or set its action to throttle rather than block. "Immediately block fake Google crawlers" does not affect us.
Sucuri Website Firewall
Check Access Control > Block User Agents and remove any entry matching LighthouseLocalBot. Sucuri has no positive allow-by-User-Agent, so if the block persists open a Sucuri support ticket quoting the User-Agent above.

Managed WordPress hosts.

Kinsta
MyKinsta > site > Bot Protection > add an exception by user agent.
WP Engine
Use the Web Rules Engine to allow the User-Agent; platform-level rate limiting may still need a support request.
SiteGround
No self-service option. Open a Help Desk ticket (Other > AI Crawlers Setup) quoting the User-Agent.
GoDaddy Managed WordPress
Website Security firewall > Block user-agents lets you allow a User-Agent string.
Flywheel
Open a support ticket quoting the User-Agent.
Other hosts
Send your host the User-Agent and ask them to allow it.

Other firewalls and CDNs.

AWS WAF
Add a rule with a string match on the User-Agent header containing LighthouseLocalBot, action Allow, priority above your block rules.
Imperva (Incapsula)
Settings > Security > Bot Access Control > Add Exception, type User agent.
Akamai Bot Manager
Define a custom bot on the User-Agent header and allow its category.

Your own server.

  1. Search the Apache .htaccess or vhost for RewriteCond %{HTTP_USER_AGENT} lines, and nginx configs for $http_user_agent checks, that return 403 for unknown or all bots.
  2. Add an exception for LighthouseLocalBot:
    RewriteCond %{HTTP_USER_AGENT} !LighthouseLocalBot [NC]

Place that line before the blocking RewriteRule. It exempts our bot from the rule that follows.

Check robots.txt too.

The homepage and robots.txt are always fetched, but the page sample honors Disallow rules for LighthouseLocalBot and falls back to *. If your robots.txt has Disallow: / under User-agent: *, add a group:

User-agent: LighthouseLocalBot
Allow: /

See what LighthouseLocalBot fetches for the full crawl scope.

What happens next.

Once the rule is in place, the next scheduled audit runs automatically and the Website Health page updates when it completes.

Still blocked?

Email us with your site URL and the message you see in the app.

support@rhetor.ai

Want to confirm it is really us? Verify the request signature.