What is technical SEO?

Technical SEO is the part of search engine optimisation that deals with how a website is built and served. It covers whether a search engine can reach your pages, read them and store them. Google sets three minimum requirements for a page: its crawler is not blocked, the page works, and the page has content Google can index. Google adds that most sites pass without trying, and that passing does not guarantee a page will be indexed.

Letting crawlers in: robots.txt and sitemaps

A crawler is the program a search engine sends to read your site. Two small files guide it.

• robots.txt sits at the top of the site, at /robots.txt, and tells crawlers which parts they may visit. It gives no protection: the standard says its rules are not a form of access authorisation. Blocking a page there does not reliably keep it out of Google either. For that, Google says to use a noindex tag or a password.
• A sitemap is a file that lists the pages on the site and can say when each was last updated. Google says a small site whose pages all link to one another may not need one, and that listing a page does not guarantee it is crawled or indexed.

A single rule in robots.txt can close a whole site to crawlers, so read the file after any rebuild. A broken file is as bad. If it returns a server error, the standard tells crawlers to treat the entire site as off limits.

One address for each page

The same page can often be reached at several addresses: over http and over https, or with tracking codes added to the end. Google groups duplicates like these and shows one of them, which it calls the canonical address. You can state your choice with a permanent redirect or with a canonical tag in the page’s code. Between an http and an https copy Google prefers https, unless something such as an invalid certificate gets in the way.

Phones and speed

Google uses the mobile version of a site for indexing and ranking, so text left off the phone version to save space may cost the page traffic. Google recommends responsive design, where one page rearranges itself to fit the screen.

Speed is judged through Core Web Vitals, which Google’s ranking systems use. The measures are set out in Core Web Vitals, explained. Google is plain about their limit: good scores do not guarantee a high position, and relevance to the search counts for more.

Structured data

Structured data is a set of labels in a page’s code that state what the page describes, such as a recipe with its ingredients and cooking time. Most of it uses a shared vocabulary from schema.org. Google uses it to understand the page, and it can make a page eligible for richer search results, though Google does not promise them. It should describe only what visitors can see on the page.

What to check

1. Run our free website health check. It loads a page the way a search engine would, reports anything telling search engines to stay away, and counts the redirects.
2. Type your address followed by /robots.txt into a browser and read what comes back.
3. In Google Search Console, use the Page indexing report and the URL Inspection tool to see which pages Google could not reach.

A new site needs these checks most. The list is in What to check before you launch a new website, and ongoing work is described on our SEO page.