# This file is used to allow crawlers to index our site. # It is a good idea to include this file otherwise a 404 is generated in the server logs as crawlers request the page # List of all web robots: http://www.robotstxt.org/db.html # # Details about Googlebot available at: http://www.google.com/support/webmasters/bin/topic.py?topic=8843 # # This file via Kentico from /Special-Pages/Robots User-agent: * Disallow: /Admin/ Disallow: /CMSDesk/ Disallow: /App_Browsers/ Disallow: /App_Code/ Disallow: /App_Data/ Disallow: /App_Themes/ Disallow: /App_WebReferences/ Disallow: /aspnet_client/ Disallow: /Bin/ Disallow: /ClientBin/ Disallow: /CMSAdminControls/ Disallow: /CMSControlsExamples/ Disallow: /CMSCSSStylesheets/ Disallow: /CMSFiles/ Disallow: /CMSFormControls/ Disallow: /CMSHelp/ Disallow: /CMSImportFiles/ Disallow: /CMSInlineControls/ Disallow: /CMSInstall/ Disallow: /CMSMasterPages/ Disallow: /CMSMessages/ Disallow: /CMSModules/ Disallow: /CMSResources/ Disallow: /CMSScripts/ Disallow: /CMSSiteManager/ Disallow: /CMSSiteUtils/ Disallow: /CMSTemplates/ Disallow: /CMSTransformations/ Disallow: /CMSWebPartLayouts/ Disallow: /CMSWebParts/ Disallow: /CommunitySite/ Disallow: /Controllers/ Disallow: /CorporateSite/ Disallow: /CorporateASPXSite/ Disallow: /EcommerceSite/ Disallow: /Global.asax Disallow: /Lib/ Disallow: /Old_App_Code/ Disallow: /Old_App_WebReferences/ Disallow: /PPG/ Disallow: /Properties/ Disallow: /Sico/ Disallow: /Views/ User-Agent: msnbot Crawl-Delay: 20