# All robots will spider the domain

User-agent: *
Disallow:/__com/
Disallow:/__css/
Disallow:/__data/
Disallow:/__download/
Disallow:/__js/
Disallow:/__mailing/
Disallow:/__scripts/