What’s a sitemap, how to build one for your site, and why
You want to help Google indexing your site!
Perhaps you have also been there. After pushing your first project online, you wonder why your website’s different pages don’t appear in the search engines like Google. It’s as if only the homepage existed.
For our site's pages to appear in our target users' searches, we need to help Google index our site. And for that, one of the things we must learn is what’s a sitemap, why we should create one, and how to do it.
Why do we create a sitemap?
When you are in a new city, you‘ll most probably need to use your maps application to know which streets to take to find a particular spot, won’t you? Well, Google also needs a map to see all the pages inside our website and not get lost. That’s why we help it by building sitemaps.
This way, Google will know which pages are inside your site and will be able to index them on its search engine. If we don’t provide any help, it will be harder to be appropriately indexed.
We create a Sitemap because it makes it easier for search engines to read the content on our website and index its pages properly. Therefore, it will improve the SEO (Search Engine Optimization) of our website.
There are two kinds of Sitemaps, though, whose purposes are different. To be indexed correctly in a search engine, we will build an XML Sitemap; and for usability purposes, we will create an HTML Sitemap.
We will see the difference between the two, but before jumping into creating a sitemap or seeing examples of the two kinds, the very first thing we need to do is to understand the structure of our website.
Understand the structure of your website.
First, we should have a clear idea of the structure of your website. There is an excellent way to make sure that you know your site's structure entirely, and that is by drawing a visual sitemap.
The example below corresponds to an uncomplicated structure of a portfolio website. As you see, we wrap each page in a box, and we represent each level of depth with a different color. Let’s look at it.
By doing the exercise of drawing a visual sitemap, you will probably spot problems on the structure, make improvements, or reassure how amazing it’s already.
Note that it’s recommended not to exceed the three navigation levels for a good SEO of the website. That means that, from the homepage, we should be able to arrive at the last depth level of our website by clicking only three times.
Now we are ready to start building our XML and HTML sitemaps.
HTML Sitemap
HTML sitemaps are made for humans, for the users to find a page in your website, or to understand its navigation easily.
In simple words, a sitemap is a list of the different pages on a website. It’s a list of the website’s URLs.
Below we see how it could look the HTML sitemap of our example (portfolio site) after styling it a little with CSS, for the users to understand the navigation of our website in the blink of an eye.
XML Sitemap
XML stands for “extensible markup language,” so it’s a kind of markup — as HTML is another kind of markup, Hypertext Markup Language. — So, XML is the way we will display information on our sitemap.
Our XML sitemap will tell search engines which content holds our website and how they can reach it. Its sole purpose is to facilitate the indexing of your site's pages in a search engine, for example, Google.
We can create an XML Sitemap and submit it to Google console — we’ll see later how to do this — so that Google will know the structure of our site and index and rank easier all its pages.
As we said, a sitemap is a list of the URLs of our website’s pages. If your website is a landing page or a single page site, the URL will be only one; then there’s no point in building an XML Sitemap. In this case, there are other ways to help google index your page, like, for example, registering your domain in the Google console through the webmaster tools provided by Google.
For that, you have to go to Google Search Console > Add Property :
How to create an XML sitemap?
First, we will create the XML file itself.
We’ll call our file sitemap.xmland this is where we will be adding our code. Let’s see how it looks with our example, the portfolio website.
We can look back at our visual sitemap above to see that all the pages we represented with a colored box, we are now listing them in our sitemap.xml.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.mysite.com/</loc>
</url>
<url>
<loc>http://www.mysite.com/about-me/</loc>
</url>
<url>
<loc>http://www.mysite.com/web-development/</loc>
</url>
<url>
<loc>http://www.mysite.com/web-development/project1.html</loc>
</url>
<url>
<loc>http://www.mysite.com/web-development/project2.html</loc>
</url>
<url>
<loc>http://www.mysite.com/web-development/project3.html</loc>
</url>
<url>
<loc>http://www.mysite.com/ux-design/</loc>
</url>
<url>
<loc>http://www.mysite.com/ux-design/case-study1.html</loc>
</url>
<url>
<loc>http://www.mysite.com/ux-design/case-study2.html</loc>
</url>
<url>
<loc>http://www.mysite.com/ux-design/case-study3.html</loc>
</url>
<url>
<loc>http://www.mysite.com/contact/</loc>
</url>
</urlset>Let’s break down each line and understand how to build it ourselves.
<?xml version=”1.0" encoding=”UTF-8"?> → That’s the XML declaration. It indicates search engines three things:
- that they’re in front of an XML file
- the version of XML (1.0)
- the character encoding used(UTF‑8)
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">→ this line indicates which standard we are using; the standard 0.90 is widely used because big search engines like Yahoo or Google support it.
The tag<urlset> acts as the container for all the sitemap URLs. The <url> tag is the parent tag we will use for each URL. Inside, we will nest the tag <loc> which will tell the location of the URL.
<loc> is the only mandatory tag inside <url> but there are others you can add if you want to provide more information to the search engines, for example: <lastmod> , <priority> , or <changefreq> . For more details about these tags, don’t hesitate to jump into reading the documentation.
That was a simple XML Sitemap example; you can see a larger one, for example, taking a look at Apple’s XML sitemap.
Validate your code.
We shouldn’t assume our sitemap is going to be perfect the first time we code it. To be sure our sitemap functions correctly, we can use online validation tools like this one.
Add your sitemap to the root.
Important note: We could place our sitemap anywhere on our site, but a sitemap affects only the parent directory's descendants. So the best place to put our sitemap is in the root folder of the project.
So let’s place the file we just created (sitemap.xml) in the root folder so that once we upload our project to the server and it is online, we will be able to access it at http://www.mysite.com/sitemap.xml .
For example, a simple project folder would look like this:
Submit your sitemap to the search engine.
In this case, we will be adding our sitemap to Google.
First, we will have uploaded our sitemap file to the server; it will be online.
Then we only have to go to Google Search Console > Sitemaps — you will find the sitemaps option on the left menu — , then we add our sitemap location and click submit.
And we are done!
Now Google will know which is our site's structure and will be able to index and rank all its pages. We’ll give it some days for the search engine to do its job, and we’ll probably find soon all our pages appearing in Google search.
Other notes
Is it wrong to have the same page listed twice?
Imagine there’s a page on our site that appears under two different categories; should we include a duplicated URL entry in our sitemap, or is that a bad practice?
In an XML sitemap, yes, that’s a bad practice. The search engine will interpret it as duplicated content and will consider you are cheating. It might even penalize the positioning our your site.
In an HTML sitemap, no, it’s not bad. As long as it helps the user understand the content better, it’s ok. Remember that the HTML sitemap is for the users, not for the search engines.
We don’t include anchor links.
When we hit an anchor link, the server doesn’t get a new request; it just retrieves the same page, and it jumps into the section defined by the id of the anchor link. A sitemap lists the individual pages on your site; therefore, you shouldn’t include anchor links.
If we add them to our sitemap, we will index the same page repeatedly. The search engine would consider this duplicated content and, once more, will not like it.
Plugins
You can build your sitemap manually if you are dealing with a small number of pages. But for extensive websites, it might be very tedious to do it manually, so you can any of the plugins out there that will generate the XML sitemap of your website for you. XML-sitemaps.com is an example of one of them.
Takeaways
- We create HTML sitemaps for humans; we create XML sitemaps for search engines.
- If we want Google to index our page, we’ll help it by creating an XML sitemap and submitting it.
- If we want to help our users understand our site, and for usability purposes, we’ll build an HTML sitemap.
- In an XML sitemap, we won’t include the anchor links; this will harm the SEO of our site.
- We will always validate our sitemap.
- For larger websites, some plugins will be helpful.
- But, for small sites, building them manually it’s fun!
