2013-09-10 WordPress
There are many opinions about content of robots.txt, that should be included to your WordPress site.
One of the optimal content is presented below:
User-agent: * Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content Allow: /wp-content/uploads/ User-agent: Mediapartners-Google Allow: / User-agent: Adsbot-Google Allow: / User-agent: Googlebot-Image Allow: / User-agent: Googlebot-Mobile Allow: / User-Agent: bingbot Allow: /
Save this code as a file called robots.txt and copy it to the root directory where your wp-config.php file is located.