Mitigate

Your website has to be AI-friendly

Arvis Zeile
Arvis Zeile
| 4 mins

People still search Google. They also ask ChatGPT, Perplexity, Claude, and Google's AI Overviews. Those systems do not click around like a person. They fetch your HTML, robots.txt, sitemap, and — if you published one — llms.txt. If who you are and what you sell is not in that text, they skip you or make it up.

This is not a replacement for ordinary SEO. Titles, a real h1, canonicals, and schema already help Google. The extra requirement is that an agent can answer from your site alone: who you are, what you sell, which pages matter, and whether it may quote you.

If those facts live only in a JavaScript app and never in the HTML, most crawlers and agents never see them.

What to have

  • A <title> and meta description that state what the company does
  • One visible h1 with real text, not a logo
  • Organisation JSON-LD (legal name, URL, contact, address, sameAs)
  • Product or SoftwareApplication JSON-LD — or a plain HTML list — with live product URLs, if you sell software
  • /llms.txt that names the company and links the pages and products that matter
  • robots.txt that allows crawling, with a sitemap that resolves
  • Canonical URLs; hreflang if you have more than one language
  • The main claims present as text in the fetched HTML

We ran this on mitigate.dev. The prompt below is the same audit, written so you can point it at any site. After the report, ask the agent what else it would change so other agents can consume your pages better.

Paste this into your agent

Paste it into Claude, ChatGPT, Cursor, or any agent that can open URLs. Turn on browsing / web fetch. Replace the URL.

Audit this public website for AI-agent readiness and on-page SEO.

Site: PASTE_URL_HERE

Rules:
- Do not guess. Fetch live URLs. If a request fails, mark that check FAIL and quote the status or error.
- Read raw HTML or text, not a screenshot.
- Quote evidence (URL + a short excerpt) for every PASS, FAIL, or WARN.
- Do not recommend a redesign, a new CMS, or a brand rewrite. Stay on machine-readable surfaces.
- Do not invent product names, addresses, or URLs. If you did not fetch it, leave it out.

Fetch:
1. The homepage HTML at the URL above.
2. {origin}/robots.txt
3. {origin}/llms.txt and {origin}/llms-full.txt (a 404 is a finding)
4. {origin}/sitemap.xml and {origin}/sitemap-index.xml
5. {origin}/.well-known/security.txt
6. RSS/Atom: any <link rel="alternate" type="application/rss+xml"> on the homepage, plus {origin}/rss.xml and {origin}/feed.xml
7. If the site is multilingual, also fetch one non-default language homepage.

Checks:

A. Crawl permission
- robots.txt exists and does not block the whole site
- The sitemap URL in robots.txt (or the sitemap you fetched) resolves and lists real page URLs
- Record Content-Signal / Content Signals if present (search, ai-input, ai-train)
- noindex on the homepage or key commercial pages is a FAIL

B. Identity an agent can cite
- <title> names the organisation or the page, not only a tagline
- meta description states what the company does, in one or two sentences
- Exactly one visible h1 on the homepage, with real text (not a logo-only h1)
- Organisation or LocalBusiness JSON-LD: legal name, url, contact, address if there is a location, sameAs
- If the company sells software: SoftwareApplication or Product JSON-LD with name, url, description — or a clear HTML list with live product URLs

C. Agent files
- llms.txt exists, is text/plain, names the company, links the important pages, and links live product URLs (not only marketing pages)
- security.txt with a contact (optional; WARN if missing)

D. Sharing and language
- Canonical URL is absolute and matches the page
- If multiple languages exist, hreflang tags point at real sibling URLs, plus x-default
- og:title, og:description, og:image (absolute URL; confirm the image loads)
- If there is a blog or news section: RSS items have title, link, and date

E. The copy is actually in the HTML
- The homepage's main claims (who, what, for whom) appear as text in the fetched HTML
- If the fetch is an empty shell and the copy only appears after JavaScript, FAIL this check

Output:
1. One-paragraph verdict.
2. Score 0–10 for agent-readiness, not visual design.
3. A table: Check | Result (PASS/FAIL/WARN/N/A) | Evidence | Fix
4. Top 5 fixes, ordered by impact, each one concrete (file to add, tag to change, URL to correct).
5. If llms.txt is missing or thin, write a ready-to-publish llms.txt based only on what you fetched.
6. If Organisation JSON-LD is missing, write a minimal JSON-LD block based only on what you fetched. Omit unknown fields.

Then stop.

Enjoy ❤️

Contact us