# What Is llms.txt and How Do You Create One? With a Copy-Paste Example

> llms.txt is a file that tells AI crawlers, in markdown, what your site is and which pages matter most. It takes about half an hour to produce. Full example and validation commands inside.

- Kanonik URL: https://algoritmaajans.com/en/blog/what-is-llms-txt-how-to-create-it
- Dil: en
- Kategori: Artificial Intelligence
- Yayın: 2026-08-17
- Güncelleme: 2026-08-17
- Yayıncı: Algoritma Ajans

---

**`llms.txt` is a file at your site root that tells AI crawlers, in markdown, what your site is and which pages matter most.** Think of it as the AI counterpart to `sitemap.xml` — with one difference: a sitemap is a machine-readable list of URLs, while `llms.txt` is a human-readable table of contents ordered by priority.

It takes about half an hour to produce. A complete example and the generation logic are below.

## Three files, three different jobs

They get conflated. They should not be.

| File | What it says | Format |
|---|---|---|
| `robots.txt` | "Where you may crawl" | Directives |
| `sitemap.xml` | "Which URLs exist" | XML, flat list |
| `llms.txt` | "What this site is, what matters" | Markdown, prioritised |
| `ai.txt` | "What you may use it for" | Directives |

`llms.txt` does not replace the others. Your sitemap stays. `llms.txt` adds a **meaning and priority** layer: if you have a thousand pages, it says which thirty actually matter.

## Why it works

When an AI crawler arrives at your site it has two problems.

**Problem one: working out what you are.** Your homepage is written in marketing language, the navigation has fifteen links, the hero section has a slogan. The crawler struggles to extract "what exactly does this company do" from that.

**Problem two: working out what is important.** The sitemap has 400 URLs; your privacy policy and your primary service page carry identical weight.

`llms.txt` solves both in one file: one paragraph of plain definition, then links grouped in priority order.

## Anatomy of the file

The structure is simple, and it is markdown:

```
# Company or Site Name

> One or two sentences of plain definition. What you do, for whom,
> in which geography. Definition language, not marketing language.

A free-form paragraph — any special notes, language coverage,
technical details like a markdown twin.

## Most Important Section

- [Page name](https://site.com/page): Short description
- [Another page](https://site.com/page2): Short description

## Second Section

- [Page](https://site.com/x): Description

## Optional

- [Less important page](https://site.com/y)
```

Four rules:

**1. Heading grouping is a priority signal.** Higher `##` sections read as more important. Order them by business priority — services near the top, legal text at the bottom.

**2. `## Optional` has special meaning.** Everything under this heading reads as "look only if needed." Privacy policy, terms of service, archive pages go here.

**3. Add a description to every link.** A short description after the colon. The crawler understands the page without visiting it.

**4. Canonical URLs only.** Redirected, noindexed or draft pages do not belong.

## Complete example

The structure of this site's `llms.txt`:

```
# Algoritma Agency

> A technology agency providing custom software development, AI-supported
> SEO, web design and AI integration training across sectors. Based in
> Türkiye. Content is published in Turkish and English.

This file is prepared for AI search engines and language models.
A markdown version of every page is available by appending .md to the URL.

## Services

- [Services](https://algoritmaajans.com/en/hizmetler): Software development, SEO, web design, AI consulting
- [AI Training](https://algoritmaajans.com/en/egitim): AI for daily work and Vibe Coding training packages
- [About](https://algoritmaajans.com/en/hakkimizda): Agency, team and working approach
- [Contact](https://algoritmaajans.com/en/iletisim): Quotes and project enquiries

## Products

- [All Products](https://algoritmaajans.com/en/urunler): Ready-to-deploy software solutions
- [AI Autonomous Content CRM](https://algoritmaajans.com/en/urunler/ai-icerik-crm)
- [Smart Appointment System](https://algoritmaajans.com/en/urunler/akilli-randevu-sistemi)
- [Online Education Software](https://algoritmaajans.com/en/urunler/online-egitim-yazilimi)
- [News Portal Software](https://algoritmaajans.com/en/urunler/haber-portali-yazilimi)

## Blog

- [Post title](https://algoritmaajans.com/en/blog/slug): Post summary

## Optional

- [Privacy Policy](https://algoritmaajans.com/en/gizlilik)
- [Terms of Service](https://algoritmaajans.com/en/kullanim-sartlari)
- [Turkish version](https://algoritmaajans.com/tr)
```

Note the ordering: services and products at the top, blog in the middle, legal text under `Optional`. That sequence is not accidental — it reflects business priority.

## llms-full.txt: the full-text version

There is a second file: `llms-full.txt`. It serves your entire content as plain text in one file, so the crawler does not have to walk page by page.

The logic: to read your blog post normally, a crawler must download HTML and strip out navigation, footer and scripts. With `llms-full.txt` it all arrives in a single request as clean text.

The difference we measured on this site is striking — same post:

| Format | Size |
|---|---|
| HTML | 139 KB |
| Markdown | 8.5 KB |

Roughly sixteen times. On the crawler side that is token cost, directly.

When is it worth it? If your content count is manageable — dozens of posts, not thousands — `llms-full.txt` makes sense. On a site with thousands of pages the file grows unwieldy; there, `llms.txt` plus per-page markdown twins is the better fit.

## Generating it dynamically

A static file can be hand-written, but nobody remembers to update it with every new post. The right approach is to generate it from your content.

In the Next.js App Router, `llms.txt` is built as a route handler — a file at `src/app/llms.txt/route.ts`. A dot in the folder name is fine; Next treats it as a literal segment.

The logic runs like this: pull published content from the database, convert it to markdown lines, return it with a `Content-Type: text/plain` header. Add a cache duration so it does not hit the database on every request.

On this site `revalidate` is 3600 seconds — refreshed hourly. A newly published post appears in `llms.txt` within an hour at most.

Three things to watch:

**Published and indexable content only.** Drafts and `noindex` pages must not appear.

**Truncate descriptions.** The first 150–160 characters of the excerpt is plenty. More just inflates the file.

**Pick one language or separate them.** A single mixed-language file confuses the crawler. Either use your primary language or open a separate section per language.

## Five common mistakes

**1. Turning your sitemap into markdown.** `llms.txt` is not a list of 400 URLs. Thirty to forty genuinely important pages is enough. The point is to signal priority; list everything and you have signalled nothing.

**2. Writing it in marketing language.** "As the sector's leading solution partner, with our customer-focused approach" tells a crawler nothing. "A Türkiye-based agency providing custom software and SEO for SMEs" tells it something.

**3. Skipping descriptions.** Listing only titles and URLs halves the information value.

**4. Letting it go stale.** A six-month-old file linking to deleted pages damages trust. Dynamic generation eliminates this entirely.

**5. Writing `llms.txt` while `robots.txt` is still closed.** The most frequent contradiction. If the crawler cannot enter the site, it cannot read `llms.txt` either. Remove the AI crawler blocks first.

## Validation

Three checks after publishing:

```bash
# 1 — reachable, and with the right content type?
curl -sI https://yoursite.com/llms.txt | grep -iE "^HTTP|content-type"
# Expected: 200 and text/plain

# 2 — is the content right?
curl -s https://yoursite.com/llms.txt | head -30

# 3 — do the links work? (lists anything not returning 200)
curl -s https://yoursite.com/llms.txt \
  | grep -oE 'https://[^)]+' \
  | while read u; do
      code=$(curl -s -o /dev/null -w '%{http_code}' "$u")
      [ "$code" != "200" ] && echo "$code  $u"
    done
```

The third check matters — an `llms.txt` full of broken links is worse than none.

## On its standard status

To be straight about it: `llms.txt` is not an official standard yet. No AI provider has documented "we read this file and use it this way."

So why do it? Three reasons.

**The cost is very low.** Half an hour of work; generated dynamically, maintenance is zero.

**There is no downside risk.** Nothing breaks; the effect on classic SEO is neutral.

**The direction is clear.** That crawlers prefer clean, structured content is an observed fact. Markdown twins and `llms.txt` are parts of the same logic.

This is not a claim that it definitely works. It is the calculation that the cost is low and the potential upside is high, so you do it. Being ready when the standard settles a year from now beats starting then.