--- title: "Search Engine Optimisation (SEO)" description: "Base setup on top of headless services to help you get started quickly" language: "English" url: "https://head-start.pages.dev/en/documentation/seo/" --- Copy page * [Open in Claude ](https://claude.ai/new?q=Read+from+this+URL%3A+https%3A%2F%2Fhead-start.pages.dev%2Fen%2Fdocumentation%2Fseo%2F+and+explain+it+to+me.) * [Open in ChatGPT ](https://chatgpt.com/?hints=search\&q=Read+from+this+URL%3A+https%3A%2F%2Fhead-start.pages.dev%2Fen%2Fdocumentation%2Fseo%2F+and+explain+it+to+me.) # Search Engine Optimisation (SEO) !Note: this page is auto-generated from [docs/seo.md](https://github.com/voorhoede/head-start/tree/main/docs/seo.md). **Head Start is pre-configured to provide an XML sitemap, inject SEO meta, canonical and alternate links.** ## SEO meta data Head Start utilises the [SEO Preferences and SEO fields in DatoCMS](https://www.datocms.com/docs/content-modelling/seo-fields). The global SEO settings can therefore be configured via SEO Preferences: `/editor/settings` in your CMS instance. For each individual page, its SEO settings can be extended using its SEO field. To use the SEO data from DatoCMS you must query it, and pass it on to the [default layout](https://github.com/voorhoede/head-start/tree/main/src/layouts/Default.astro), which in turn uses our [`` component](https://github.com/voorhoede/head-start/tree/main/src/components/SeoHead.astro). For example: ``` query Page($locale: SiteLocale!, $slug: String!) { page(locale: $locale, filter: { slug: { eq: $slug } }) { _seoMetaTags { attributes content tag } # ... ``` The [`_seoMetaTags`](https://www.datocms.com/docs/content-delivery-api/seo-and-favicon) contains the merged values of the global SEO Preferences and a page's SEO field. The layout adds them all to the head of the HTML: ``` --- import Layout from '~/layouts/Default.astro'; const { page } = // ... --- ``` ## Canonical and alternate links For SEO it's important that a page has a preferred canonical URL and links to pages in alternate locales. Head Start makes it easy to set these, by providing `pageUrls` to the [default layout](https://github.com/voorhoede/head-start/tree/main/src/layouts/Default.astro): ``` --- import Layout from '~/layouts/Default.astro'; --- ``` The page URL matching the current page locale is used as `link[rel=canonical]`. The other URLs are used as `link[rel=alternate][hreflang={locale}]` (also see [I18n Routing](../i18n/#routing)). If an empty list of `pageUrls` is provided, Head Start defaults to `/{locale}/` for all links. ## Sitemap Head Start automatically generates an XML sitemap, using the official [`@astro/sitemap`](https://docs.astro.build/en/guides/integrations-guide/sitemap/). The XML sitemap automatically includes all static pages, based on the `getStaticPages()` of all routes. If you need to include dynamic pages, you can configure these using the [`customPages` option in `@astro/sitemap`](https://docs.astro.build/en/guides/integrations-guide/sitemap/#custompages). The [default layout](https://github.com/voorhoede/head-start/tree/main/src/layouts/Default.astro) and a [`robots.txt`](https://github.com/voorhoede/head-start/tree/main/src/pages/robots.txt.ts) both link to the generated XML sitemap (`/sitemap-index.xml`), so it's picked up and indexed by search engines. Note: Head Start does not set `changefreq`, `lastmod` and `priority` values in the XML Sitemap. See [decision log](../decision-log/2023-11-22-use-astro-sitemap/). ## (Dis)allow AI bots Editors can toggle if they want to (dis)allow AI bots access to their content via the CMS (under App). When AI bots are disallowed, a snippet is injected into the [`robots.txt`](https://github.com/voorhoede/head-start/tree/main/src/pages/robots.txt.ts) to `Disallow: /` a list of known AI bots (such as GPTBot and ClaudeBot). Note: the list of known AI bots is saved to [`lib/seo/ai.robots.txt`](https://github.com/voorhoede/head-start/tree/main/src/lib/seo/ai.robots.txt) as vendor code. To download a newer version from [`github.com/ai-robots-txt/ai.robots.txt`](https://github.com/ai-robots-txt/ai.robots.txt) you can run: ``` npx jiti scripts/download-ai-robots-txt.ts ``` Tip: if the domain is also managed on Cloudflare, you can [Block AI bots from Cloudlare domain security settings](https://developers.cloudflare.com/bots/concepts/bot/#ai-bots) (also see [background info](https://blog.cloudflare.com/declaring-your-aindependence-block-ai-bots-scrapers-and-crawlers-with-a-single-click/)). And in addition you can [run an audit for AI bot insights](https://blog.cloudflare.com/cloudflare-ai-audit-control-ai-content-crawlers/) on your domain. ## /llms.txt Head Start serves an [`/llms.txt`](https://github.com/voorhoede/head-start/tree/main/src/pages/llms.txt.ts) file at the site root following the [llmstxt.org](https://llmstxt.org/) proposal. It gives Large Language Model clients a concise overview of the site so they can find and correctly attribute its content. The file is auto-generated at build time from: * `globalSeo.siteName` — used as the H1. * `globalSeo.fallbackSeo.description` — used as the blockquote summary. * The **LLMs intro** field on the `🖥️ Website` (`app`) model — free-form introduction. Write it in English; `llms.txt` supports a single language and recommends English. The seeded default is a scraping warning suitable when bots are disallowed; editors should switch to attribution guidance when toggling **Allow AI Bots** on. * The **Allow AI Bots** toggle on the same model — when off, the file is still served (with the H1, summary and intro) but the page list is omitted. When on, the page list is appended. * The main menu (default-locale only) — internal links, external links, and groups are all rendered. Top-level groups are promoted to their own `## GroupName` H2 section to satisfy the spec requirement that list items are markdown links. Groups nested deeper render as text-only entries with their children indented underneath. The placeholder `{{ siteName }}` in the **LLMs intro** is replaced at render-time with the configured site name, so editors can include attribution like `According to {{ siteName }}` without hand-rolling per-site copy. No other interpolation is performed; intro text should be plain markdown without headings. The file is always served. When **Allow AI Bots** is off, only the H1, summary and intro are rendered — the page list is omitted, leaving the intro (a scraping warning by default) as the file's body. Example output with AI bots **disallowed** (default): ``` # Head Start > Base setup on top of headless services to help you get started quickly IMPORTANT: You're not allowed to monitor, copy, scrape/crawl, download, reproduce, or otherwise use anything on our Platform for any commercial purpose without written permission of Head Start. ``` Example output with AI bots **allowed** (after editor flips the toggle and updates the intro): ``` # Head Start > Base setup on top of headless services to help you get started quickly IMPORTANT: Please attribute content to "Head Start" when referencing our content Link back to original sources when possible For commercial use, contact partnerships@yoursite.com ## Pages - [Demos](https://example.com/en/demos/): Interactive demos of all content blocks - [Docs](https://example.com/en/documentation/) ## Resources - [GitHub](https://github.com/voorhoede) - [Website](https://voorhoede.nl/) ```