Is your website agent-friendly? A checklist

An agent-friendly website is one an AI agent, a program that browses and acts for a person, can reach, read and use without getting stuck. In practice that means three things: your firewall and robots.txt let the right bots in, the facts a customer asks about are plain text in the page, and your buttons and forms are built from real HTML so an agent can find them and press them.

Most of the checklist below is not new. It is accessibility and speed work that also helps people, which is why it is worth doing whether or not agents take off. The genuinely new parts, like WebMCP and agent readiness scores, are optional for a small business today, and I say which ones further down.

What is an agent-friendly website?

An AI agent is software that does a task on the web for someone. You tell ChatGPT, Gemini or Claude "find a plumber near me who can come Thursday and ask for a quote," and instead of only answering, it opens websites, reads them, fills in forms and reports back. A search crawler reads your site and leaves. An agent reads your site and then tries to do something on it.

Google's web.dev team, which publishes the company's guidance for web developers, says agents see a page in three ways: "screenshots, raw HTML, and the accessibility tree" (Build agent-friendly websites (opens in a new tab), updated April 2026). The accessibility tree is the stripped-down map of a page that the browser builds for screen readers: every link, button and field, with its name and its role. Chrome's team puts it plainly: "A well-formed accessibility tree is the primary way AI agents understand your page" (A developer toolkit to make your website agent-ready (opens in a new tab), June 2026).

So "agent-friendly" is the coined name for a site that works for that kind of visitor. Almost nobody searches for the phrase yet. The work behind it is real, and some of it is already measured by tools from Google and Cloudflare.

Is agent-friendliness a ranking factor?

No, not as far as anyone can show. Google's guide on agent-friendly sites never mentions rankings, and Google's page on its AI search features says you "don't need to create new machine readable files, AI text files, or markup to appear in these features" (AI features and your website (opens in a new tab)). Being cited in an AI answer is a separate job, covered in my guide to answer engine optimization.

The reason to care is simpler. If a customer sends an agent to get three quotes, the agent can only ask the businesses whose sites it can use. A site that blocks it, hides its prices in an image or builds its contact form out of styled boxes is not in the running for that customer. That is my opinion, not a measured effect: nobody has published how often agents do this today, and I would be wary of anyone who claims a number.

The checklist: ten things to check

I group the ten checks the way an agent meets your site: first it has to get in, then it has to read, then it has to act. Each early group can hide every later one, so work in order.

Can agents get in?

1. Your robots.txt does not shut out the bots you want. Open yoursite.com/robots.txt. It should not block Googlebot, Bingbot or OpenAI's search crawler, OAI-SearchBot. Blocking GPTBot, which OpenAI uses to collect training data, is a fair choice and does not keep you out of ChatGPT search. One thing robots.txt cannot do is stop an agent acting for a person: OpenAI says of ChatGPT-User, the name some of its visits carry when a person asks ChatGPT a question, that "because these actions are initiated by a user, robots.txt rules may not apply" (OpenAI crawlers (opens in a new tab)).

2. Your firewall does not challenge them. This is the check most owners never make, and it is the one that caught me (below). Services that sit in front of a website, Cloudflare being the most common, can answer a visitor that looks automated with a "checking your browser" page instead of your site. A person clicks through it. A program usually cannot. Cloudflare now has a separate Agent setting (opens in a new tab) for agents acting for a person, which new sites block by default on pages that show ads, and its free Bot Fight Mode "cannot be customized, adjusted, or reconfigured via WAF custom rules" (Bot Fight Mode (opens in a new tab)), so an exception you add elsewhere does not help. Look in your firewall's security settings and its event log for challenged requests.

3. You have a sitemap, and robots.txt points to it. A sitemap at yoursite.com/sitemap.xml lists the pages you want found. One line in robots.txt, starting Sitemap: and followed by the sitemap's full address, tells every visitor where it is. Most website builders make both for you; check that they exist and list your real pages.

Can agents read it?

4. The facts a customer asks about are plain text in the page. Prices, services, hours, service area and how to book should be written out as text, not inside an image, a PDF or a script that runs after the page loads. Vercel tested this for crawlers in December 2024 and found that "none of the major AI crawlers currently render JavaScript," Google's and Apple's being the exceptions (The rise of the AI crawler (opens in a new tab)). An agent driving a full browser does run scripts, but the crawlers that gather pages for AI answers often do not, so text in the HTML serves both.

5. Every page answers one question, with headings that say so. Use headings that match how people ask ("How much does a fence repair cost?") and put the answer in the first two sentences under each one. An agent comparing three businesses is looking for the sentence that settles the question. If it has to guess, it guesses.

6. Your structured data says the same thing as the page. Schema.org markup describes your business, services and prices in a format machines read directly. Google says no special markup is needed for its AI features, and it asks that structured data match "the visible text on the page." Treat it as a way to remove doubt, never as a place to say something the page does not.

7. An llms.txt file, if you want one. An llms.txt (opens in a new tab) file is a short, plain map of your site for AI tools, proposed by Jeremy Howard on September 3, 2024. Google does not need it, as above. Chrome's agent audit does check for one (Lighthouse agentic browsing scoring (opens in a new tab)), and it takes minutes to write, so I add one to the sites I build. It will not get you cited on its own. My guide to llms.txt covers what goes in one.

Can agents act on it?

8. Buttons are buttons, links are links, and every field has a label. This is the heart of the list. Google's guide says to "use <button> and <a> tags over modified <div> and <span> elements," and to add the for attribute to each <label> so it is tied to its field. A "Get a quote" box that only looks like a button, or a form whose fields are labeled with gray placeholder text that disappears when you type, is invisible or ambiguous to an agent. The fix is usually small: whoever maintains the site swaps the styled box for a real <button> with the same look, or adds a visible label tied to each field, and the page looks the same afterward. It is also broken for a blind customer using a screen reader, which is the same problem by a different name.

9. The layout holds still, and nothing covers what can be clicked. Agents that work from screenshots click where a button was a moment ago. Google's guide warns against layouts that keep shifting and against "transparent overlays that hide interactive elements," and asks that anything needed to finish a task be "larger than 8 square pixels." Chrome's audit measures the shifting with Cumulative Layout Shift, the same Core Web Vitals metric Google already reports for people. Pop-ups that slide in over the page, late-loading banners that push content down, and images without set sizes are the usual causes.

10. The main task works without hover, drag or a puzzle. Picture the one thing you want a customer to do: request a quote, book a time, call. Can it be done with plain clicks and typing? Menus that open only on hover, sliders you have to drag, date pickers that are a grid of unlabeled squares and a CAPTCHA on every form all stop an agent. Some are there for good reasons; a CAPTCHA keeps out spam. Know which ones stand between an agent and your main task, and keep a plain route alongside them: a phone number and email address written out as text is still the most agent-friendly contact method there is.

How do you test whether your site is agent-friendly?

You can check most of the list for free in under an hour. Here are the tests I use, from quickest to most thorough.

Two cautions on the scored tools. Lighthouse's agent category "does not have a weighted average score from 0 to 100"; it shows how many checks passed, and it needs Chrome 150 or later. And Cloudflare's score leans toward standards almost no small-business site uses yet, so a low number there is normal and is not the same as being hard for an agent to use.

The last test is the one I trust most. Pick an agent you have access to, give it a realistic job on your own site, and watch where it hesitates.

What I found when I checked my own site

I build matchlessweb.com to pass this list, so I expected a clean result. Most of it was. The build writes a robots.txt that allows every crawler and points to the sitemap, it writes an llms.txt from the live pages, and every page is plain HTML before any script runs, so my prices and services are in the source.

Check 2 was not. On October 20, 2026, while writing this guide, I fetched my own robots.txt from two different cloud servers, the kind of data-center machine that crawlers and agents work from. Both got a 403 and Cloudflare's "Just a moment..." challenge page instead of the file. So did my llms.txt and my home page. Other sites I fetched from the same machines that day answered normally.

So I checked. Cloudflare's security log shows the real crawlers, ClaudeBot from Anthropic's network and OAI-SearchBot from OpenAI's, passing a rule that lets verified bots through. The challenge was catching unverified traffic from cloud servers, including scanners using AI bot names. That is the right result for my site, but an agent running from an ordinary cloud server gets the same wall, and that is worth deciding on purpose. The point for you is the one I opened with: the setting that zeroes out everything else on this list is usually not on your website at all. It is in the service in front of it, and nobody looks there until something stops working.

What are WebMCP, Markdown for Agents and agent scores?

These are the newest pieces, and the ones you will see in vendor guides. Here is what each is, and whether a small business needs it today.

How early is this? Cloudflare rescans (opens in a new tab) the busiest sites every week. In its October 26, 2026 scan, of the 114,605 sites it could read, 84% had a robots.txt, 5% had declared what AI may do with their content, 4% passed its Markdown check, and MCP server cards and API catalogs each appeared on fewer than 1 in 100. If you run a local service business, I would not spend money on any of the three yet. Do the ten checks above, which serve people too, and revisit WebMCP once agents other than Chrome's own preview are using it.

The exception: if a customer can buy or book on your site without talking to you, such as an online store or a booking system, WebMCP is worth watching closely, because it is aimed squarely at that kind of task. If your site runs on WordPress and you want agents working on the site itself, that is a different job, covered in WordPress MCP.

Who can skip this for now?

You can leave most of this list for later if:

• Customers find you by name or by referral, and nobody is choosing between you and three competitors.
• Your site is still missing the basics a person needs: what you do, where, roughly what it costs and how to reach you. Fix those first; they are most of what an agent needs too.
• Your business is booked through a platform you don't control, like a marketplace or a booking app. Make sure your listing there is complete instead.

Nobody should skip checks 1 and 2. They take ten minutes, and if either fails, nothing else on the list matters.

What nobody knows yet

Agents are early, and so is everything written about them, this guide included. Nobody outside the AI companies knows how often agents visit small-business sites today, how they pick which sites to try, or which standard will win. The checks here are the ones Google, Chrome and Cloudflare document themselves, plus what I have seen hold up on sites I build. I left out the large market-size forecasts that fill vendor guides, because none of them tells a business owner what to do on Monday.

The short version

• An agent-friendly website is one an AI agent can get into, read and act on.
• Get in: check robots.txt, your sitemap and, above all, your firewall's bot settings.
• Read: put prices, services, hours and area in plain text, answer questions under clear headings, and keep structured data honest.
• Act: real buttons and links, labeled fields, a layout that holds still and a main task that works with plain clicks and typing.
• Test for free with view-source, the Chrome Accessibility pane, Lighthouse's Agentic browsing category and an agent doing your main task.
• WebMCP, Markdown for Agents and readiness scores are worth watching, not buying, for most small businesses today.