In order for a website to write a good WordPress robots file, there are still some WordPress pages to study.
Here are the categories that must be banned:
Disallow the inclusion of WordPress system files:
Disallow:/wp-admin/
Disallow:/wp-content/
Disallow:/wp-includes/
Do not include repeat content, author pages, dynamic pages and other WordPress spam pages:Disallow: /trackback/
Disallow: /author/
Disallow: //feed Disallow: /function.
Disallow: /?s=*
Disallow: //?s=\
Disallow: /wp-*.php
Combining the above, we can write a complete WordPress site robots file like this:
User-agent: *
Disallow:/wp-admin/
Disallow:/wp-content/
Disallow:/wp-includes/
Disallow: /trackback/
Disallow: /author/
Disallow: //feed Disallow: /function.
Disallow: /?s=*
Disallow: //?s=\
Disallow: /wp-*.php
Disallow:/DotBot
Disallow:/SemrushBot
Disallow:/Uptimebot
Disallow:/MJ12bot
Disallow:/MegaIndex.ru
Disallow:/ZoominfoBot
Disallow:/Mail.Ru
Disallow:/SeznamBot
Disallow:/BLEXBot
Disallow:/ExtLinksBot
Disallow:/aiHitBot
Disallow:/Researchscan
Disallow:/DnyzBot
Disallow:/spbot
Disallow:/YandexBot
Disallow:/404.html
sitemap:https://bkzzz.com/sitemap.xml
Do not allow dotbot to grab yandexbot, these are unnecessary spiders.