Today we're pleased to announce an update to the Sitemaps Protocol, in collaboration with Google, and Yahoo! This update should help many new sites adopt the protocol by increasing our flexibility on where sitemaps are hosted.

Essentially, the change allows a webmaster to store their sitemap files just about anywhere, using a reference in the Robots.txt file to establish a trusted relationship between the sitemap file and the domain or folder.

Here's how it works: Say you run a web site like MSN.com, which has a bunch of sub domains like health.msn.com, travel.msn.com and moneycentral.msn.com. And, due to a technical requirement, you would like to host all of your sitemaps in one location like sitemaps.msn.com. Until now the protocol did not support this scenario, each sitemap would have needed to be hosted directly under the domain it described. This update now introduces support for this scenario, with the requirement that you simply include a reference to the sitemap in your Robots.txt file. For example, moneycentral.msn.com/robots.txt would need to include this line:

Sitemap: http://sitemaps.msn.com/index_moneycentral.msn.com.xml

The catch is that all the URLs in the sitemap file all need to be within the same domain as the robots.txt file (i.e. moneycentral.msn.com/* in this example). Note that this applies equally for sitemap index files and for compressed files.

Here are a few other useful notes about our implementation:

  • We support multiple "Sitemap:" references in your robots.txt files
  • We recommend you limit the size of your robots.txt file to less than 1 MB
  • If multiple sitemaps for a domain include the same URL with conflicting metadata (i.e. priority, change frequency, etc), we will disregard the metadata and just look at the URL.
  • Individual sitemap files should never be larger than 10 MB when uncompressed. This includes all sitemap file formats: XML, RSS and Text.
  • You can upload your sitemap in our Webmaster Tools
  • You can ping us with updates to your sitemap using our Ping URL: 
    http://webmaster.live.com/ping.aspx?siteMap=[Your sitemap URL]

This change comes directly from feedback we received from webmasters, thank you for helping us improve our product! If you have any additional feedback or questions, please check out our Sitemap Discussion forum.

-- Fabrice Canel, Program Manager, Live Search Crawler