An llms.txt generator builds a valid llms.txt file for your website in about a minute. You enter your site name, a short description, and your key page links, and the tool outputs the markdown file in the exact structure AI systems expect. Then you upload it to your site root at example.com/llms.txt, and you are done.
That is the whole process. No signup, no plugins, no coding. If you are short on time, try the free llms.txt generator, paste the output into a file named llms.txt, and upload it to the top-level folder of your website. This guide shows you the exact steps, what belongs in each section, copy-paste examples for different site types, and the mistakes that make most llms.txt files useless.
What an llms.txt generator actually does
llms.txt is a proposed convention — first published by Jeremy Howard of Answer.AI in September 2024 at llmstxt.org — for a plain markdown file that lives at the root of a website. It gives AI systems a short, curated map of the site's most important pages instead of forcing them to crawl and guess.
The word "proposed" matters. llms.txt is not an official standard like robots.txt (which is an IETF specification, RFC 9309). It is a community convention that is gaining traction because it solves a real problem: full websites rarely fit in a model's context window, and raw HTML full of navigation, ads, and scripts converts to text poorly. Markdown is clean and token-efficient, which is why the proposal uses it.
A generator does one useful job: it turns your site details into that markdown file with the correct structure, so you do not have to memorize the format. What follows is how to do it in 60 seconds.
Create your file in 60 seconds
Here is the exact workflow:
- Open the generator. Go to CrawlGuards' free llms.txt generator. It runs in your browser and asks for nothing — no email, no account.
- Enter your site name and homepage URL. Use the canonical domain (the version with or without
wwwthat you actually use). This becomes the H1 heading — the only strictly required element of the file. - Write a one-to-three sentence summary. Say what the site is, who it serves, and what someone can do with it. This becomes the blockquote summary directly under the H1.
- Add your key pages in groups. Think in 2 to 5 sections — for example, "Getting Started", "Documentation", "Pricing", "Blog". Under each, add your most useful pages with a one-sentence description per link. Leave out archives, tag pages, and thin content. This file is a curated map, not a sitemap.
- Copy the output and save it as
llms.txt. Upload it to the root of your domain so it is reachable atyourdomain.com/llms.txt. Visit that URL in your browser to confirm it loads.
That is genuinely all there is to it. The rest of this article is about making the file good instead of just present — which is where most people stop.
What a good llms.txt file contains
The proposed format has only a few parts. Getting each one right takes five minutes of thought and makes the file far more useful.
The H1: exactly one
The first line of the file is a single H1 with your site or brand name: # Acme Plumbing. There should be exactly one H1 in the file. This is the file's title and the only element the proposal marks as mandatory.
The blockquote summary
Directly under the H1 comes a blockquote (lines starting with >) with one to three sentences: what the site is, who it serves, and what an AI can accomplish by reading the listed resources. Write it for a reader who has never seen your site. Bad: "We are the best in the business." Good: "Acme Plumbing is a licensed plumbing company in Austin, TX. Homeowners can get service pricing, book appointments, and read troubleshooting guides."
H2 sections with link lists
After the summary, group your links under H2 headings. Each link is a markdown link followed by a colon and a one-sentence description:
## Services
- [Drain Cleaning](https://acmeplumbing.example/drain-cleaning): Pricing and booking for residential drain cleaning in Austin.
- [Emergency Repairs](https://acmeplumbing.example/emergency): 24/7 emergency plumbing response times and service area.
Rules that keep the file clean: use absolute URLs (full https:// addresses, not relative paths), describe what the page answers rather than what it is called, and keep the whole file lean. A few kilobytes is the right size. If your file is pushing tens of thousands of words, you are dumping, not curating — and that is a separate file (see the llms-full.txt section below).
Three copy-paste examples
Adapt one of these to your site type. They are intentionally short — that is the point.
Example 1: personal blog
# Priya's Money Notes
> Priya's Money Notes is a personal finance blog for young professionals in the US. Readers can learn budgeting basics, compare high-yield savings accounts, and read honest reviews of money apps.
## Start Here
- [Budgeting 101](https://example.com/budgeting-101): A beginner's guide to building your first budget that actually sticks.
- [Emergency Funds Explained](https://example.com/emergency-fund): How much to save and where to keep it.
## Reviews
- [High-Yield Savings Accounts Compared](https://example.com/hysa-compared): Current rates and fees for popular savings accounts.
- [Budgeting Apps Tested](https://example.com/budgeting-apps): Hands-on reviews of five budgeting apps.
## About
- [About Priya](https://example.com/about): Who writes this blog and how it is funded.
Example 2: local business
# Acme Plumbing
> Acme Plumbing is a licensed plumbing company serving Austin, TX. Homeowners can see service pricing, book appointments, and read troubleshooting guides.
## Services
- [Drain Cleaning](https://example.com/drain-cleaning): Pricing and booking for residential drain cleaning.
- [Water Heaters](https://example.com/water-heaters): Repair and replacement options with upfront pricing.
- [Emergency Repairs](https://example.com/emergency): 24/7 emergency response times and service area.
## Company
- [Pricing](https://example.com/pricing): Flat-rate price list for common jobs.
- [Service Area](https://example.com/service-area): Neighborhoods served in the Austin metro.
- [Contact](https://example.com/contact): Phone, hours, and booking form.
Example 3: documentation site
# WidgetJS Docs
> WidgetJS is an open-source JavaScript UI library. Developers can find installation steps, the full API reference, and copy-paste code examples.
## Getting Started
- [Installation](https://example.com/docs/install): Install WidgetJS with npm, CDN, or script tag.
- [Quick Start](https://example.com/docs/quickstart): Build your first widget in under 10 minutes.
## API Reference
- [Core API](https://example.com/docs/api): Complete reference for all WidgetJS methods and options.
- [Events](https://example.com/docs/events): Event names, payloads, and handler examples.
## Resources
- [Examples Gallery](https://example.com/examples): Working demos with source code.
- [Changelog](https://example.com/changelog): Version history and migration notes.
Notice what is missing from all three: navigation pages, legal boilerplate links, archives, author pages, and anything that does not help a newcomer understand the site. If you want the deeper technical background, our complete llms.txt guide walks through the format, benefits, and implementation in more detail.
Common mistakes that make llms.txt useless
Most llms.txt files fail for boring, fixable reasons. Here are the five I see most often:
1. Wrong location. The file must be at the domain root: example.com/llms.txt. Putting it at example.com/blog/llms.txt or example.com/docs/llms.txt defeats the purpose — the convention only works because the location is predictable.
2. The 10,000-line dump. Auto-generating the file from your sitemap with every URL included turns a curated map into noise. An AI reading 400 links with no curation learns nothing. Pick your 10 to 30 most useful pages. Everything else is what llms-full.txt is for — or nothing at all.
3. Descriptions that describe nothing. "Click here to learn more" and "Our services page" waste the one sentence you get per link. Say what the page answers: who it is for and what question it resolves.
4. Stale links. A file full of 404s or redirects to deleted pages is worse than no file, because it actively misleads. This happens when the file is generated once and forgotten. (See "When to regenerate" below.)
5. Including things that should not be there. Admin pages, login URLs, staging subdomains, internal dashboards, and anything behind a paywall do not belong in a public map of your site. Keep it to public, useful pages.
How to validate your file
Run through this checklist after uploading:
- It loads at the root. Visit
yourdomain.com/llms.txtin a fresh browser tab. You should see the markdown text, not a 404 page. - It is served as plain text. The file should render as raw text in the browser, not wrapped in your site's HTML theme. If your CMS serves it inside a page template, fix that — the convention assumes raw markdown.
- Exactly one H1. The file starts with one
# Titleline and contains no others. - The blockquote summary exists. One to three sentences directly under the H1, written for a stranger.
- Every link resolves. Click through each link (or run a link checker). No 404s, no chains of redirects, no placeholder URLs.
- URLs are absolute and consistent. Full
https://addresses, all on the same canonical domain you actually use. - It is short. If a human cannot skim it in two minutes, trim it.
What about llms-full.txt?
The proposal also describes an optional companion file, llms-full.txt, served at the same root path. Where llms.txt is the map, llms-full.txt is the territory: the full markdown text of your key pages concatenated into one document, so an agent gets everything in a single fetch.
Most sites do not need it. It is worth considering if you run a documentation site, a knowledge base, or an API reference where detailed, verbatim content is the product. For a blog or a local business, the short file is enough. If you do build one, keep it to your most important pages and regenerate it whenever those pages change — a stale full-text file is a liability, not an asset.
llms.txt and robots.txt work as a pair
People often ask whether llms.txt replaces robots.txt. It does not. They do opposite jobs:
- robots.txt tells crawlers what they are not allowed to fetch. It is the file where you permit or block specific AI crawlers — for example, allowing search-oriented bots while blocking training crawlers.
- llms.txt tells AI systems what your site is and where the good content lives. It assumes the crawler is already allowed and gives it a head start.
A sensible setup is both: a robots.txt that deliberately names the AI crawlers you want (OpenAI's, Anthropic's, Perplexity's — each provider documents its own bots and the directives they honor, e.g. Anthropic's crawler documentation) plus an llms.txt that points the allowed ones at your best pages. If you need to build the first half, this free robots.txt generator handles it, and this comparison goes deeper on how the two files differ.
When to regenerate
An llms.txt file rots quietly. Put a reminder on your calendar for these moments:
- Quarterly, at minimum — open the file, click the links, and fix anything stale.
- After a redesign or URL restructuring, when old paths may 404.
- When pricing, services, or core offerings change, since those pages are usually the ones you linked.
- When you publish a new flagship guide that deserves a place in the map — and when you retire one, remove it.
Regenerating takes another 60 seconds with the generator, so there is no reason to let it go stale. A current, honest file beats an impressive, outdated one every time.
Frequently asked questions
Is llms.txt an official web standard?
No. It is a proposed convention published in September 2024, not an IETF or W3C standard, and no major search engine or AI provider requires it. Treat it as a low-cost, good-practice addition — like a well-written meta description — not as a ranking requirement.
Will llms.txt improve my Google rankings?
There is no verified evidence that it does, and you should be skeptical of anyone who guarantees it. What it honestly offers is a cleaner way for AI systems and agents to discover your best content, since markdown is far cheaper for them to process than full HTML. That is a reasonable bet with real upside potential, but it is not an SEO shortcut.
Do I still need robots.txt if I have llms.txt?
Yes — more than ever. robots.txt controls which crawlers may access your site at all, including AI training crawlers. llms.txt only helps the crawlers you have already allowed. One governs access, the other guides attention.
Where exactly does the file go?
At the root of your domain: https://yourdomain.com/llms.txt. It must be reachable without redirects through subfolders, served as plain text, and use your canonical domain.
How long should the file be?
Short. Think one screen to a few screens of text: a title, a short summary, and a few sections of curated links. If it takes more than a couple of minutes to skim, it is too long. Full content belongs in the optional llms-full.txt, not here.
Can I just auto-generate it from my sitemap?
You can, but you usually should not publish the raw output. A sitemap lists everything; llms.txt should list the best. Use a sitemap as a starting inventory, then curate ruthlessly down to the pages a newcomer actually needs.
Sixty seconds of setup, five minutes of thought about what belongs in it, and a quarterly check to keep it fresh — that is the entire cost of a good llms.txt file. Start with the free llms.txt generator, upload the file to your site root, and you will be ahead of most of the web on this one.