# All robots will spider the domain User-agent: * Disallow:/__com/ Disallow:/__data/ Disallow:/__mailing/ Disallow:/__scripts/