# bots

how this service identifies itself to websites, what it honors, and how to opt out.

> draft: not reviewed by a lawyer. do not rely on it before launch.

## who we are

Hosted runs and page reads identify themselves with the user agent `orsel-reader` and `orsel-runtime`, and sign their requests with HTTP message signatures (the Web Bot Auth draft, Ed25519). Our public keys are served at `/.well-known/http-message-signatures-directory`. If that address answers 404, this deployment has no signing key configured and sends unsigned requests.

## the crawler

Our search index is built by our own crawler, which identifies itself as `orsel-bot`. It reads `robots.txt` before every site, waits at least 2 seconds between requests to a site (longer if you set `Crawl-delay`), fetches public pages only, never logs in, never sends cookies, and follows redirects, checking the robots.txt and crawl delay of the site it lands on. It stores the page text and a vector of its meaning, and links back to the original page in results.

To keep it out, add this to your `robots.txt`:

```
User-agent: orsel-bot
Disallow: /
```

Disallow only part of a site the usual way (`Disallow: /private`). When a page becomes disallowed, it is removed from the index the next time the crawler visits it.

## what we honor

- `robots.txt` for page reads.
- A site owner's policy once the domain is claimed: rate limits, blocked agents, blocked hosted runs, per-action prices.
- HTTP 402. A priced action stops and reports the price; nothing is paid on your behalf.
- Rate limit responses (429, `Retry-After`).

## what we never do

- Solve or route around captchas, bot detection or two-factor prompts. The run stops and hands over to a person.
- Run logged-in sessions from our servers by default. Those run in the user's own browser.
- Follow redirects off the declared origins of a recipe.

## opt out

Claim your domain (a DNS TXT record or a well-known file) and set the owner policy to block hosted runs or specific agents. To report abuse or ask for removal, write to hi@egeuysal.com.
