Before starting any SEO project, there is one thing we always need to check first: Is my website getting indexed properly?
If your webpages aren’t getting indexed, or even crawled by search engines, it is pointless to do optimization.
We all know that such technical issues can cause headaches (we really know!), but not all technical SEO implementation is that hard to understand or execute.
An XML sitemap is one of those seemingly technical but actually simple SEO implementations; it is an easy-to-use tool that can solve lots of problems on crawlability and indexation.
Here, we give you insights and tips on how to best implement your XML sitemap.
So what exactly is an XML sitemap?
To put it simply, an XML sitemap is a list of webpages on your website. You can include the URLs that you want Google to index in your XML sitemap and improve your website’s crawlability as a result.
So how does an XML sitemap help search engines crawl your website content faster and easier? As the image above illustrates, Google can find page 9 immediately with an XML sitemap. In contrast, Google will have to jump through 5 other pages to find it if the sitemap does not exist.
And that is why the XML sitemap is important for SEO. If a search engine cannot crawl and index your webpage, how can it send traffic to you?
So, when doing SEO, a websites’ crawlability should always be the first priority.
Do I need an XML sitemap?
Is the XML sitemap a must if we want our websites to be crawled and indexed by Google? Well technically, the answer is no. As suggested by Google, if a website has its webpages properly linked, then their web crawlers should be able to discover most of the website.
However, it is never harmful to make Google’s job easier. Google particularly recommends websites that are new, large, and with webpages that are not well-linked to include an XML sitemap, so that Google will not overlook some of its webpages.
Including a URL in our XML sitemap doesn’t necessarily mean that Google will crawl your page as the XML sitemap only works as a suggestion to Google. Still, submitting the sitemap to Google can increase the chance that your webpage gets crawled. So again—it’s no harm to make Google’s job easier!
Tip to Creating an SEO XML Sitemap: Only include URLs that you want Google to index
The name “XML sitemap” may give you an impression that a sitemap should be a roadmap of the whole website. But that is wrong—you don’t need to include every page of your website in the sitemap. Instead, you should only include webpages that you want Google to crawl.
When Google’s web crawler comes to a website, the crawler has a crawl budget to limit the number of webpages that it will crawl. You won’t want Google’s crawler to waste the crawl budget for visiting webpages that you don’t need indexed. Instead, you will hope the crawler focuses on those optimized webpages.
Including URLs in the XML sitemap indicates to Google that those webpages are more important than pages that are not included. It is a signal to Google to prioritize crawling those webpages ahead of others. As a result, the sitemap can help your website to efficiently use crawl budget.
So what kind of webpages should be excluded from the XML sitemap? Here are our recommendations:
- Pages requiring log-in
- Utility pages (e.g.: review forms, wish lists, etc.)
- Duplicate pages
- Paginated pages
- URLs with parameters or session IDs
- Any URLs that are redirected
- Any URLs that no longer work (broken URL)
- Pages disallowed by robots.txt
- Pages with noindex
How do I submit my XML sitemap to Google?
Once you have created your XML sitemap and uploaded it to your domain (your URL will end with “/sitemap.xml”), you can submit the sitemap to Google via Google Search Console.
Once you login to Google Search Console, you can find Sitemaps on the left, under Index.
After clicking Sitemaps, you can input your website’s sitemap URL in the Add a new sitemap column, and submit to Google.
And it’s done! It may take a few days for Google to process your submission and crawl your website with the assistance of an XML sitemap. You can check the status in Sitemaps.