# robots.txt # Allow all robots to spider the domain User-agent: * Disallow: # Disallow directories User-agent: * Disallow: /htdocs_new/ sitemap: http://www.military-research.co.uk/sitemap.xml