Amazonbot

Amazonbot is a web crawler used by Amazon to index search results, allowing the Alexa AI Assistant to provide more accurate answers to user questions. It crawls websites to gather information like titles, images, keywords, and links, which helps power Alexa’s AI-powered search results. While Amazonbot’s visitation frequency varies based on factors like search demand, it generally respects robots.txt rules to limit access to certain pages

Amazonbot crawls your website to enhance its services, such as improving Alexa’s responses and gathering data for indexing. You can control its access via the robots.txt file

Add the following lines to your robots.txt file at the root of your server:

text
User-agent: Amazonbot
Disallow: /

This will instruct the Amazonbot to not crawl any pages on your site.

Using .htaccess

You can also block the Amazonbot in your .htaccess file using mod_rewrite:

text
# Block Amazonbot
<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteCond %{HTTP_USER_AGENT} (Amazonbot) [NC]
  RewriteRule (.*) - [F,L]
</IfModule>

This will return a 403 Forbidden response to the Amazonbot.

Verify the bot

To ensure you are blocking the actual Amazonbot and not a bot pretending to be Amazonbot, you can verify it by doing a reverse DNS lookup on the bot’s IP address:

text
$ host 12.34.56.789
789.56.34.12.in-addr.arpa domain name pointer 12-34-56-789.crawl.amazonbot.amazon.
$ host 12-34-56-789.crawl.amazonbot.amazon
12-34-56-789.crawl.amazonbot.amazon has address 12.34.56.789

If the reverse DNS resolves to amazonbot.amazon, it is the official Amazonbot.