Our scanner, and how to opt out

What it is

LiftMCP-Scanner checks how ready a website is for AI agents: whether it publishes the standard files that agents look for, and whether its pages can be read and used by them. It is run by LiftMCP Ltd, a company registered in England and Wales (company number 17177294).

We use it to measure readiness across the web, and anyone can use it to check a site on our website.

How to recognise it

Every request it makes carries this User-Agent. The number is the version of our checks, and changes when they do.

Mozilla/5.0 (compatible; LiftMCP-Scanner/5; +https://www.liftmcp.com/scanner)

Requests come from Amazon Web Services in London (eu-west-2). The addresses vary from visit to visit.

What it asks for

A visit is a few dozen requests, each address at most once. It reads your robots.txt first, then your home page and a fixed set of public files, such as:

• /robots.txt, /sitemap.xml and /llms.txt
• files under /.well-known/ that agents look for, such as mcp.json, api-catalog and oauth-authorization-server
• a few common pages, such as a contact or cart page

Most sites publish few of these files, so a visit usually leaves a handful of 404s in your logs. That is expected.

If your site's /.well-known/mcp.json names an MCP server, the scanner connects to it as an AI agent would: it lists the server's tools and calls one with no arguments, to check that it answers.

It reads only public addresses. It does not log in, submit forms, or try to get past a CAPTCHA or a block.

How often

We scan a list of popular websites, drawn from Google's Chrome UX Report, in occasional passes. Each pass visits a site once. Separately, a person can ask for a scan of one site on our website; that is a single visit, made on their behalf.

How to opt out

With robots.txt. Our automated scanning follows your robots.txt. To keep it away from your whole site, add:

User-agent: LiftMCP-Scanner
Disallow: /

It also follows the rules you set for all bots under User-agent: *, when no group names it. If robots.txt disallows only some paths, it does not request those paths.

By email. Write to support@liftmcp.com with your domain, and we will add it to our do-not-scan list. That covers every address under the domain, and it stays in place.

A scan that a person asks for on our website is treated like a search engine's user-requested fetch, and does not read robots.txt.