Sitemaps XML Protocol Endpoint for Search Engines

Since MvcSiteMapProvider keeps track of your application's URLs, it is the logical place to use as a data source for exporting the URLs for search engine indexing. By default, this feature is enabled and you get it for free. Simply navigate to /sitemap.xml to see its result.

If you don't need this feature, it can be disabled by setting the MvcSiteMapProvider_EnableSitemapsXml setting to false. If using external DI, it can be disabled by removing the following line from MvcSiteMapProviderConfig.cs

    //// Register the Sitemaps routes for search engines
    //XmlSiteMapController.RegisterRoutes(RouteTable.Routes);

Large Sitemaps

Whenever a sitemap exceeds 35,000 nodes the XmlSiteMapController will automatically split your sitemap into a sitemap index file (/sitemap.xml) which references sub-sitemaps (/sitemap-1.xml, /sitemap-2.xml, etc.) as described on http://www.sitemaps.org/protocol.php.

Note that we chose the number 35,000 as an average cap because we don't actually measure the physical size of the sitemap result, which cannot exceed 10MB.

GZip Compression

Whenever a client sends an HTTP request header with Accept-encoding set to a value of gzip or deflate, the XmlSiteMapResult class (which is also used internally in the XmlSiteMapController) will automatically compress the sitemaps XML using GZip compression.

Customizing the Sitemaps XML Result

Note: The technique described here can only be done when using an external DI container.

If you need to customize the result (for example, if you need to combine the results of multiple SiteMap objects into one Sitemaps XML result), you can do this easily by building your own custom controller and using the overloads of the Create method of XmlSiteMapResultFactory to change the output.

Let's say you have followed the instructions on Multiple Sitemaps in One Application page to add 2 SiteMaps to your application, and your SiteMap cache keys are now "mainNavigation" and "footerNavigation". You decide you want to combine these two sitemap files into one result so you can submit them to search engines all at once.

The default behavior of XmlSiteMapController is to only include the URLs that match the SiteMapCacheKey for the current HTTP request (as assigned by ISiteMapCacheKeyGenerator). But we can override the default behavior in our custom controller. This is how we would do that.

[AllowAnonymous]
public class CustomXmlSiteMapController
    : Controller
{
    public CustomXmlSiteMapController(
        IXmlSiteMapResultFactory xmlSiteMapResultFactory
        )
    {
        if (xmlSiteMapResultFactory == null)
            throw new ArgumentNullException("xmlSiteMapResultFactory");
        this.xmlSiteMapResultFactory = xmlSiteMapResultFactory;
    }

    private readonly IXmlSiteMapResultFactory xmlSiteMapResultFactory;


    public ActionResult Index(int page = 0)
    {
        return xmlSiteMapResultFactory.Create(page, new string[] { "mainNavigation", "footerNavigation" });
    }
}

As you can see, we are using an overload of XmlSiteMapFactoryResult that accepts multiple SiteMap cache keys, and we are passing in the 2 specific ones that we want. The SiteMap cache keys can be thought of like the name of a SiteMap instance.

XmlSiteMapResult will automatically filter out any URLs that are exactly the same in the "mainNavigation" and "footerNavigation" SiteMaps.

Now, we need to disable the standard routes that MvcSiteMapProvider uses as described near the top of this page. Note that if you are using a DI modules-only package, this line may be somewhere else in your application's composition root.

    // MvcSiteMapProviderConfig.cs file

    //// Register the Sitemaps routes for search engines
    //XmlSiteMapController.RegisterRoutes(RouteTable.Routes);

Then we need to register our own routes to point to our new custom controller, one for the sitemap index, and one for the paged sitemaps. Note you can change the URL to whatever scheme you prefer.

routes.MapRoute(
    name: "SiteMap",
    url: "sitemap.xml",
    defaults: new { controller = "CustomXmlSiteMap", action = "Index", page = 0 }
);

routes.MapRoute(
    name: "SiteMap-Paged",
    url: "sitemap-{page}.xml",
    defaults: new { controller = "CustomXmlSiteMap", action = "Index", page = 0 }
);

We generally don't need to touch the DI configuration because IXmlSiteMapResultFactory can be resolved by the default MvcSiteMapProvider registration logic. However, there may be a need to register it explicitly in some DI configurations.

Now if you run the application, you will see the results of both SiteMap instances in a single combined XML result that conforms to the Sitemaps protocol.