Technical SEO Checklist for New Websites: Crawlability, Canonicals, Sitemaps, and Redirects

Overview

This guide gives you a reusable technical SEO checklist for new websites. It is not a list of every possible ranking factor. Instead, it covers the setup items that most often determine whether search engines can discover, interpret, and consolidate your content correctly.

For a new site, technical SEO is mostly about removing friction. Can search engines reach the pages you want indexed? Are duplicate URLs consolidated? Does your sitemap reflect the right canonical version of each page? Do old URLs resolve cleanly after a redesign or migration? If these basics are wrong, content quality and on-page optimization have less room to work.

Keep one principle in mind: every important page should have a single preferred URL, a consistent internal linking path, a valid server response, and clear signals about whether it should be indexed. Most technical SEO issues happen when those signals conflict.

Core setup goals

• Make important pages crawlable: avoid accidental blocks in robots.txt, noindex tags, authentication layers, or JavaScript-dependent navigation that hides discoverable links.
• Choose one canonical version per page: prevent duplicate paths, mixed protocol versions, trailing slash inconsistencies, parameter duplicates, and alternate hostnames from competing.
• Publish a clean XML sitemap: include indexable canonical URLs only, and keep excluded, redirected, broken, or staging URLs out.
• Handle redirects deliberately: preserve equity and usability during domain changes, CMS changes, URL renames, and content consolidation.
• Align hosting and delivery settings: HTTPS, CDN rules, caching, and rewrite behavior should reinforce your preferred URL structure, not create variants.

If you are launching a site from scratch, pair this guide with a broader website launch checklist for SEO, analytics, forms, and indexing. If your setup also involves domain cutover, DNS changes, or SSL work, see how to launch a website on a custom domain and how to set up SSL on a website.

Checklist by scenario

Use the scenario that matches your current project. The underlying checks overlap, but the failure points are slightly different depending on whether you are launching a fresh site, redesigning an existing one, or moving hosting and domains.

Scenario 1: Brand-new website launch

This is the cleanest starting point because you are not inheriting old URLs. The main risk is publishing with development settings still active.

• Confirm the site is publicly reachable. Remove password protection, IP allowlists, or temporary maintenance gates from the production environment.
• Check robots.txt. Make sure it does not block the entire site or key directories. A development rule left in place can stop crawling almost completely.
• Check page-level directives. Remove accidental noindex or nofollow tags from templates, CMS settings, or SEO plugins.
• Standardize your preferred URL format. Decide on HTTPS, preferred hostname, trailing slash behavior, lowercase path conventions, and whether index filenames resolve or redirect.
• Force a single host and protocol. Redirect HTTP to HTTPS and redirect the non-preferred hostname to the preferred one. This reduces duplicate versions.
• Add self-referencing canonical tags. Each indexable page should generally point to its own preferred URL unless there is a clear reason to consolidate elsewhere.
• Generate an XML sitemap. Include only canonical, indexable, live URLs with useful content. Exclude search results, login pages, admin paths, filtered duplicates, and utility pages.
• Link key pages in navigation or body content. Important pages should not be orphaned. If internal links do not exist, crawl discovery becomes weaker and slower.
• Return correct status codes. Live pages should resolve with 200 responses. Broken pages should not masquerade as 200 responses with error text.
• Create a simple 404 page. It should help users recover without trapping search engines in endless soft error experiences.
• Submit the sitemap in search tooling. This is not a substitute for crawlable architecture, but it helps discovery and monitoring.

Scenario 2: Redesign on the same domain

Redesigns often break SEO not because the domain changed, but because templates, navigation, or URL patterns changed silently.

• Crawl the old site before launch. Export a list of existing URLs, titles, canonicals, and status codes so you know what needs to be preserved or redirected.
• Map old URLs to new URLs. If paths change, create one-to-one redirects wherever practical. Redirect to the closest equivalent page, not just the homepage.
• Retain high-value pages where possible. Avoid unnecessary path changes for pages that already earn traffic, links, or conversions.
• Review internal links after template changes. Menus, breadcrumbs, related content blocks, and footer links should point directly to final URLs, not redirected ones.
• Review canonical output on all templates. Category, tag, blog, product, and landing page templates often behave differently after a redesign.
• Update the sitemap immediately after launch. Remove retired URLs and make sure new canonical URLs are represented.
• Watch for mixed content and HTTPS inconsistencies. Redesigns can reintroduce old asset paths or hardcoded HTTP links. This can affect trust, rendering, and crawl consistency.

Scenario 3: Migration to new hosting, CMS, or infrastructure

Migration adds operational risk. DNS, caching layers, edge rules, and origin settings can all change how search engines see the site.

• Reduce moving parts where possible. Avoid changing domain, CMS, URL structure, and content architecture at the same time unless you have a strong reason.
• Test redirects on a staging mirror. Validate common paths, trailing slash behavior, uppercase requests, legacy filenames, media paths, and known linked pages.
• Verify headers at the edge and origin. CDNs, reverse proxies, and managed hosting platforms can override canonical behavior indirectly through rewrites, cache rules, or redirect loops.
• Check DNS and cutover timing. After switching hosts or CDN providers, confirm the intended environment is serving the live domain. See the DNS propagation checker guide if you need a practical verification process.
• Verify SSL before and after cutover. Certificate issues can trigger browser warnings, indexing confusion, or accidental fallback to alternate hostnames. If needed, review SSL setup, redirects, and mixed content fixes.
• Rebuild or review robots.txt and sitemap generation. Different platforms handle these files differently, and defaults are not always production-ready.
• Monitor response codes after launch. Look for spikes in 404s, redirect chains, 5xx errors, and soft 404 behavior.

Scenario 4: Website builder, CMS, or static site workflow changes

Different platforms expose different levels of control, and changing tools can alter your technical SEO defaults.

• Check what the platform controls automatically. Some systems auto-generate canonicals and sitemaps, while others require manual templates or plugins.
• Review URL customization limits. If the builder or CMS imposes directory structures, make sure they are acceptable before launch.
• Test pagination, tags, categories, and archives. These often create duplicate or low-value pages if left unmanaged.
• Validate image, script, and stylesheet paths. Asset handling can affect rendering, performance, and crawl consistency.
• Review rendering dependencies. If primary navigation or core content depends heavily on client-side JavaScript, confirm that link discovery still works reliably.
• Choose the right platform for your SEO control needs. If you are still deciding between approaches, compare tradeoffs in website builder vs CMS vs static site generator.

What to double-check

This is the section to revisit right before launch or right after a major change. These are the details that frequently look correct in one place but fail in another.

Crawlability and indexation signals

• Robots.txt is not blocking important sections. Remember that robots.txt controls crawling, not indexing on its own. It should not be used as a shortcut for pages that need cleaner handling.
• Noindex is used intentionally. Utility pages, thank-you pages, internal search pages, and duplicate filtered views may be candidates. Core pages should not inherit noindex from a template.
• Important links are HTML links. Navigation hidden behind forms, script events, or inaccessible components may reduce discoverability.
• Pagination and archives are sensible. Avoid creating infinite crawl paths through filters, calendars, sort combinations, or faceted URLs.

Canonical consistency

• Canonical tags match the final resolved URL. If the canonical points to HTTP, a non-preferred hostname, or a redirecting URL, fix it.
• Self-referencing canonicals are present on standard pages. This helps reinforce the preferred URL version.
• Do not canonicalize unrelated pages together. A canonical is a consolidation signal, not a substitute for redirects or thin content cleanup.
• Internal links, canonicals, sitemap entries, and redirects all agree. Conflicting signals create ambiguity.

XML sitemap setup

• The sitemap includes only indexable canonical pages. Exclude redirects, noindex pages, duplicate parameter URLs, 404 pages, and staging paths.
• The sitemap updates when content changes. A stale sitemap is not fatal, but it becomes less useful for monitoring and discovery.
• Large sites segment sitemaps logically. Posts, products, categories, or image sitemaps can be split if needed, but keep the structure understandable.
• The sitemap is reachable and returns the correct content type. Do not assume the file exists just because a plugin or platform says it does.

Redirect hygiene

• There is one redirect hop whenever possible. Chains waste time and can complicate debugging.
• Temporary redirects are not used for permanent URL moves. For lasting changes, use permanent logic that reflects the intended canonical destination.
• Retired pages are handled intentionally. Redirect to the closest useful replacement when one exists. If not, a true 404 or 410 may be more honest than forcing an irrelevant destination.
• Protocol, hostname, and path normalization are covered. Redirect logic should catch common alternate versions consistently.

Hosting, CDN, and performance alignment

• HTTPS is enforced everywhere. This includes the apex domain, subdomain variants used publicly, and asset delivery where applicable.
• CDN rules do not create duplicate URLs. Edge rewrites, cache keys, or alternate hostnames can introduce inconsistent indexing paths if not planned carefully.
• Performance basics are in place. Fast hosting, sensible caching, image optimization, and stable rendering support crawl efficiency and user experience. For a deeper pass, use the Core Web Vitals checklist for hosted websites and the CDN setup guide for websites.

Common mistakes

Most launch-related SEO problems are not advanced. They are ordinary implementation oversights that survive into production because nobody checked the site end to end.

• Leaving development blocks in place. A global disallow rule, password wall, or noindex setting is one of the most common preventable errors on new websites.
• Submitting a sitemap full of non-canonical URLs. If your sitemap includes redirects, duplicates, or query parameter pages, it weakens the clarity of your setup.
• Canonicalizing everything to the homepage. This does not solve duplicate content problems and can suppress valid pages.
• Redirecting every removed URL to one generic page. Users and search engines both benefit from topical relevance. Broad homepage redirects are usually a weak fallback.
• Ignoring trailing slash and case variants. If /About, /about, and /about/ behave differently, your site can generate unnecessary duplicates.
• Relying on JavaScript alone for navigation. Modern rendering is better than it used to be, but important links should still be easy to discover in the delivered HTML when possible.
• Changing URLs during redesign without a redirect map. Even a small brochure site can lose continuity if old paths vanish overnight.
• Forgetting image, PDF, or document paths during migrations. These assets may have earned links or support useful user journeys.
• Launching multiple live versions. It is surprisingly easy to leave a staging subdomain, preview environment, or alternate hostname exposed and indexable.
• Treating technical SEO as a one-time launch task. CMS updates, plugin changes, CDN reconfiguration, and template edits can all reintroduce old problems.

If your launch also involves registrar, nameserver, or mail-related work, separate those tasks carefully from SEO changes. Domain cutovers often affect more than just web traffic. For adjacent operational planning, see the domain transfer checklist and best domain registrars compared.

When to revisit

Technical SEO setup should be reviewed whenever the underlying delivery environment changes. A practical schedule is to revisit this checklist before launch, one to two weeks after launch, before seasonal planning cycles, and any time you change tools, templates, or infrastructure.

Revisit this checklist when:

• You launch a new website or microsite.
• You redesign templates or navigation.
• You move to new cloud web hosting, managed hosting, or a different CDN.
• You change your CMS, website builder, or static site workflow.
• You connect a new domain, migrate a domain, or change DNS.
• You merge sections of the site, retire content, or rewrite URL structures.
• You introduce multilingual sections, subfolders, or subdomains.
• You notice indexing drops, crawl anomalies, redirect issues, or traffic loss after deployment.

A practical 15-minute review routine

1. Open robots.txt and confirm it is production-safe.
2. Test the homepage and a few key pages for status code, canonical, and meta robots output.
3. Check that HTTP and non-preferred hostnames redirect to the single preferred version.
4. Load the XML sitemap and verify that the listed URLs are canonical and live.
5. Click through navigation and confirm important pages are linked directly.
6. Test a sample of old URLs if the site was redesigned or migrated.
7. Review edge cases: trailing slash variants, uppercase requests, removed pages, and query parameters.
8. Document any exceptions. Technical SEO is easier to maintain when redirect rules, canonical intent, and exclusion logic are written down.

The goal is not perfection on day one. The goal is clarity. Search engines respond better when your site consistently tells the same story through its URLs, links, tags, sitemaps, and server behavior. If you treat this as a living checklist rather than a launch-only task, you reduce avoidable SEO regressions every time your website evolves.