Helping AI search tools find and understand your website starts with good technical SEO. The following files and settings support this, although not all are essential or specifically designed for AI.
- robots.txt: Controls which parts of your website compliant crawlers can access. Check that your rules allow public content to be crawled by relevant search bots, including OpenAI’s OAI-SearchBot for ChatGPT search. Also check that your hosting, firewall or security settings do not unintentionally block legitimate search crawlers.
User-agent: * Disallow: /admin/ Sitemap: https://www.mywebsite.com/sitemap.xml - XML sitemap: Lists the pages you want search engines to discover. Keep it updated and include its location in your robots.txt file.
- Schema structured data: Adds standardised information about your business, articles or products. Use relevant markup that accurately reflects the visible content. No special AI schema is required.
- FAQ markup: A type of structured data for questions and answers displayed on a page. It is optional; publishing useful, clearly written answers is more important.
- llms.txt: An optional file intended to guide AI tools towards useful website content. It is not a requirement for AI visibility, and Google does not use it for search.
- Accessible page content: Keep important information available as readable HTML text, rather than only inside images or content that requires user interaction to appear.
- Other SEO settings: Check descriptive page titles, canonical tags, internal links and accidental noindex directives that could prevent important pages appearing in search.
These technical foundations support discovery, but useful, original and reliable content remains essential. No file or markup can guarantee inclusion in AI-generated answers.