What are sitemaps?

Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.

Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow crawlers that support Sitemaps to pick up all URLs in the Sitemap and learn about those URLs using the associated metadata. Using the Sitemap protocol does not guarantee that web pages are included in search engines, but provides hints for web crawlers to do a better job of crawling your site.

The Sitemaps Protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. A Sitemap is an XML file that lists the URLs for a site. It allows webmasters to include additional information about each URL: when it was last updated, how often it changes, and how important it is in relation to other URLs in the site. This allows search engines to crawl the site more intelligently. Sitemaps are a URL inclusion protocol, and complement robots.txt a URL exclusion protocol.

Sitemaps are particularly beneficial in situations

* when it is difficult for users to access all areas of a website through a browseable interface. In these cases, a search engine can’t find these pages. For example, a site with a large “archive” or “database” of resources that aren’t well linked to each other (if at all), only accessible via a search form.
* where webmasters use rich AJAX or Flash, and search engines can’t navigate through to get to the content.

The webmaster can generate a sitemap containing all accessible URLs on the site and submit it to search engines. Since MSN, Yahoo, and Google use the same protocol now, having a sitemap would let the three biggest search engines have the updated pages information.

Sitemaps supplement and do not replace the existing crawl-based mechanisms that search engines already use to discover URLs. By submitting Sitemaps to a search engine a webmaster is only helping that engine’s crawlers to do a better job of crawling their site(s). Using this protocol does not guarantee that your webpages will be included in search indexes nor does it influence the way that pages are ranked by a search engine.

Here is a free sitemap generator.