---
title: "LighthouseLocal Bot: Crawler Documentation | Lighthouse Local"
description: "How the LighthouseLocal Bot crawler works: user-agent strings, what it fetches during a site audit, how to block it in robots.txt, and how to verify its requests via Web Bot Auth."
source: "https://www.lighthouselocal.ai/bot"
---

# LighthouseLocal Bot: The Crawler Behind Your Audit.

This page documents LighthouseLocalBot for site owners and AI assistants: what it is, the user-agent strings it sends, what it fetches, how to block it, and how to verify its requests.

On this page

- [What LighthouseLocal Bot is](#what-it-is)
- [Key facts](#key-facts)
- [User-agent strings](#user-agents)
- [What it fetches](#what-it-fetches)
- [Blocking LighthouseLocalBot](#blocking)
- [Verifying requests](#verification)
- [Questions about this crawler](#contact)

## What LighthouseLocal Bot is.

Operated by [Rhetor](https://rhetor.ai), LighthouseLocal Bot powers Lighthouse Local's website audits and AI-search-visibility checks. It crawls a site only when the site owner, or their agency, requests an audit of that site. It is not a broad web crawler, and it does not build a search index or train models.

## Key facts.

- **Operator:** [Rhetor](https://rhetor.ai)
- **Purpose:** Powers Lighthouse Local's website audits and AI-search-visibility checks.
- **Trigger:** Only when the site owner, or their agency, requests an audit of that site — not a broad web crawler.
- **Robots.txt token:** LighthouseLocalBot
- **Verification:** Every request is cryptographically signed per Web Bot Auth (RFC 9421).

## User-agent strings.

Every request carries one of these `User-Agent` strings, depending on which product triggered the audit.

Base, No Product Specified

```
LighthouseLocalBot/1.0 (+https://www.lighthouselocal.ai/bot)
```

[AI Visibility Tracker Audits](https://www.lighthouselocal.ai/how-it-works#tracker)

```
LighthouseLocalBot/1.0 (lighthouse; +https://www.lighthouselocal.ai/bot)
```

[Beacon Audits](https://www.lighthouselocal.ai/beacon)

```
LighthouseLocalBot/1.0 (seo-report; +https://www.lighthouselocal.ai/bot)
```

## What it fetches.

LighthouseLocalBot fetches a site's homepage, `robots.txt`, `llms.txt`, and sitemap for every audit, plus a sample of up to 12 additional pages (4 concurrent requests) for deeper analysis.

The homepage and `robots.txt` are fetched regardless of `Disallow` rules, since the requested audit depends on them. The page sample honors `robots.txt` `Disallow` rules, matched against `LighthouseLocalBot` and falling back to `*` when no bot-specific group exists.

## Blocking LighthouseLocalBot.

To stop the page-sample crawl on your site, add this to your `robots.txt`:

```
User-agent: LighthouseLocalBot
Disallow: /
```

Lighthouse Local customer whose audit is being blocked? See [Allow our crawler through your firewall](https://www.lighthouselocal.ai/help/allow-our-crawler).

## Verifying requests.

Every request is cryptographically signed per Web Bot Auth (RFC 9421 HTTP message signatures), so you can confirm a request claiming to be LighthouseLocalBot actually is one. The public key directory is published at [www.lighthouselocal.ai/.well-known/http-message-signatures-directory](https://www.lighthouselocal.ai/.well-known/http-message-signatures-directory).

## Questions about this crawler.

Email [support@rhetor.ai](mailto:support@rhetor.ai) with any questions about this crawler.

## Sitemap

- [Markdown sitemap](https://www.lighthouselocal.ai/sitemap.md)